Questions about the real mode in kvm/qemu

Questions about the real mode in kvm/qemu

[Li Qiang]

[-- Attachment #1: Type: text/plain, Size: 819 bytes --]

Hi Paolo and all,

There are some question about the emulation for real mode in kvm/qemu. For
all the
question I suppose the 'unstrict guest' is not enabled.

1. how the protected mode CPU emulate the real mode? It seems it uses vm86,
however, vm86 is not available in x86_64 CPU? So what's the
'to_vmx(vcpu)->rmode.vm86_active' here vm86 means?

2. Does the guest's real mode code run directly in native CPU? It seems
'vmx->emulation_required' is also be false, it the vmx_vcpu_run will do a
switch to guest.

3. How the EPT work in guest real mode? The EPT is for GVA->GPA->HPA,
however there is no GVA, seems the identity mapping does something. But
there also some confusion for me. For example the real mode uses CS*4 + IP
to address the code.  Who does this calculation? In the kernel emulator?

Thanks,
Li Qiang

[-- Attachment #2: Type: text/html, Size: 1033 bytes --]

Re: Questions about the real mode in kvm/qemu

[Maxim Levitsky]

On Thu, 2019-09-26 at 15:52 +0800, Li Qiang wrote:
> Hi Paolo and all,
> 
> There are some question about the emulation for real mode in kvm/qemu. For all the 
> question I suppose the 'unstrict guest' is not enabled. 
> 
> 1. how the protected mode CPU emulate the real mode? It seems it uses vm86, however, vm86 is not available in x86_64 CPU? So what's the 'to_vmx(vcpu)->rmode.vm86_active' here vm86 means?
> 

As far as I know it, modern intel's cpus support so called unrestricted guest mode, which allows guest to be basically in any mode,
as long as EPT paging is used (that is guest can be in real mode with
no paging, but EPT has to be enabled).
The 'vm86_active' is probably lefover support for cpus that don't support EPT and/or the unrestricted guest mode,
where KVM tried to use the good old vm86 mode to
for real mode virtualization.


> 2. Does the guest's real mode code run directly in native CPU? It seems 'vmx->emulation_required' is also be false, it the vmx_vcpu_run will do a switch to guest.

Same as above

> 
> 3. How the EPT work in guest real mode? The EPT is for GVA->GPA->HPA, however there is no GVA, seems the identity mapping does something. But there also some confusion for me. For example the real
> mode uses CS*4 + IP to address the code.  Who does this calculation? In the kernel emulator? 

EPT sits underneath the guest's paging mode, which in case of real mode is 1:1 mapping.
Thus CS<<4 + IP would be the guest physical address and it will be looked up in the EPT to translate to the real physical address.



Best regards,
	Maxim Levitsky

Re: Questions about the real mode in kvm/qemu

[Li Qiang]

[-- Attachment #1: Type: text/plain, Size: 2130 bytes --]

Maxim Levitsky <mlevitsk@redhat.com> 于2019年9月26日周四 下午4:31写道：

> On Thu, 2019-09-26 at 15:52 +0800, Li Qiang wrote:
> > Hi Paolo and all,
> >
> > There are some question about the emulation for real mode in kvm/qemu.
> For all the
> > question I suppose the 'unstrict guest' is not enabled.
> >
> > 1. how the protected mode CPU emulate the real mode? It seems it uses
> vm86, however, vm86 is not available in x86_64 CPU? So what's the
> 'to_vmx(vcpu)->rmode.vm86_active' here vm86 means?
> >
>
>
Hi Maxim,

Thanks for your kind reply.



> As far as I know it, modern intel's cpus support so called unrestricted
> guest mode, which allows guest to be basically in any mode,
>

Right, but I also want to know the secret when the 'unstrict guest' is
disabled. So I suppose the 'unstrict guest' is  not enabled for these
questions.


> as long as EPT paging is used (that is guest can be in real mode with
> no paging, but EPT has to be enabled).
> The 'vm86_active' is probably lefover support for cpus that don't support
> EPT and/or the unrestricted guest mode,
> where KVM tried to use the good old vm86 mode to
> for real mode virtualization.
>
>
> > 2. Does the guest's real mode code run directly in native CPU? It seems
> 'vmx->emulation_required' is also be false, it the vmx_vcpu_run will do a
> switch to guest.
>
> Same as above
>
> >
> > 3. How the EPT work in guest real mode? The EPT is for GVA->GPA->HPA,
> however there is no GVA, seems the identity mapping does something. But
> there also some confusion for me. For example the real
> > mode uses CS*4 + IP to address the code.  Who does this calculation? In
> the kernel emulator?
>
> EPT sits underneath the guest's paging mode, which in case of real mode is
> 1:1 mapping.
>

It seems when the 'unstrict guest' is enabled, there is no identity mapping
table.

Thanks,
Li Qiang



> Thus CS<<4 + IP would be the guest physical address and it will be looked
> up in the EPT to translate to the real physical address.
>
>
>
> Best regards,
>         Maxim Levitsky
>
>

[-- Attachment #2: Type: text/html, Size: 3175 bytes --]

Re: Questions about the real mode in kvm/qemu

[Maxim Levitsky]

On Thu, 2019-09-26 at 16:52 +0800, Li Qiang wrote:
> 
> 
> Maxim Levitsky <mlevitsk@redhat.com> 于2019年9月26日周四 下午4:31写道：
> > On Thu, 2019-09-26 at 15:52 +0800, Li Qiang wrote:
> > > Hi Paolo and all,
> > > 
> > > There are some question about the emulation for real mode in kvm/qemu. For all the 
> > > question I suppose the 'unstrict guest' is not enabled. 
> > > 
> > > 1. how the protected mode CPU emulate the real mode? It seems it uses vm86, however, vm86 is not available in x86_64 CPU? So what's the 'to_vmx(vcpu)->rmode.vm86_active' here vm86 means?
> > > 
> > 
> 
> Hi Maxim,
> 
> Thanks for your kind reply.
> 
>  
> > As far as I know it, modern intel's cpus support so called unrestricted guest mode, which allows guest to be basically in any mode,
> 
> Right, but I also want to know the secret when the 'unstrict guest' is disabled. So I suppose the 'unstrict guest' is  not enabled for these questions.
>  
> > as long as EPT paging is used (that is guest can be in real mode with
> > no paging, but EPT has to be enabled).
> > The 'vm86_active' is probably lefover support for cpus that don't support EPT and/or the unrestricted guest mode,
> > where KVM tried to use the good old vm86 mode to
> > for real mode virtualization.
> > 
> > 
> > > 2. Does the guest's real mode code run directly in native CPU? It seems 'vmx->emulation_required' is also be false, it the vmx_vcpu_run will do a switch to guest.
> > 
> > Same as above
> > 
> > > 
> > > 3. How the EPT work in guest real mode? The EPT is for GVA->GPA->HPA, however there is no GVA, seems the identity mapping does something. But there also some confusion for me. For example the
> > real
> > > mode uses CS*4 + IP to address the code.  Who does this calculation? In the kernel emulator? 
> > 
> > EPT sits underneath the guest's paging mode, which in case of real mode is 1:1 mapping.
> 
> It seems when the 'unstrict guest' is enabled, there is no identity mapping table.

If you mean to ask if there is a way to let guest access use no paging at all, that is access host physical addresses directly,
then indeed there is no way, since regular non 'unrestricted guest' mode required both protected mode and paging, and 'unrestricted guest' requires
EPT.
Academically speaking it is of course possible to create paging tables that are 1:1...


Best regards,
	Maxim Levitsky

Re: Questions about the real mode in kvm/qemu

[Paolo Bonzini]

On 26/09/19 09:52, Li Qiang wrote:
> Hi Paolo and all,
> 
> There are some question about the emulation for real mode in kvm/qemu.
> For all the 
> question I suppose the 'unstrict guest' is not enabled. 
> 
> 1. how the protected mode CPU emulate the real mode? It seems it uses
> vm86, however, vm86 is not available in x86_64 CPU? So what's the
> 'to_vmx(vcpu)->rmode.vm86_active' here vm86 means?

vm86 mode is available in 64-bit CPUs; it is not available in 64-bit
mode (EFER.LMA=1).

Once KVM places the processor in VMX non-root mode (VMLAUNCH/VMRESUME),
the processor can leave 64-bit mode and run in vm86 mode.  And that's
exactly what KVM does on processors without unrestricted guest support.

In that mode, KVM will start trapping all exceptions and send them to
handle_rmode_exception.  This in turn will either emulate privileged
instructions (for #GP or #SS) or inject them as realmode exceptions.

However...

> 2. Does the guest's real mode code run directly in native CPU? It seems
> 'vmx->emulation_required' is also be false, it the vmx_vcpu_run will do
> a switch to guest.

... as soon as the guest tries to enter protected mode, it will get into
a situation which is not real mode but doesn't have the segment
registers properly loaded with selectors.  Therefore, it will either
hack things together (enter_pmode) or emulate instructions until the
state is accepted even without unrestricted guest support.

The "hacking things together" part however does not work for big real
mode (where you enter 32-bit mode, load segment registers with a flat 4G
segment, and go back to real mode with the 4G segments).  In big real
mode, therefore, KVM cannot use vm86 and will keep emulating (slowly -
up to 1000x slower than unrestricted guest).

> 3. How the EPT work in guest real mode? The EPT is for GVA->GPA->HPA,
> however there is no GVA, seems the identity mapping does something. But
> there also some confusion for me. For example the real mode uses CS*4 +
> IP to address the code.  Who does this calculation? In the kernel emulator? 

Right, and in fact the CR0.PG and CR0.PE bits must be 1 when running in
VMX non-root mode with unrestricted guest disabled.

Therefore, KVM places a 4 KiB identity map at an address provided by
userspace (with KVM_SET_IDENTITY_MAP_ADDR).  When 1) EPT is enabled 2)
unrestricted guest isn't 3) CR0.PG=0, KVM points CR3 to it and runs the
guest with CR0.PG=1, CR4.PAE=0, CR4.PSE=1.  This CR4 setup enables 4 MiB
huge pages so that a 4 GiB identity map fits in 4 KiB.  This is the only
case where EPT is enabled but CR3 loads and stores will trap to the
hypervisor.  This way, the guest CR3 value is faked in the vmexit
handler, but the processor always uses the identity GVA->GPA mapping.

Paolo

Re: Questions about the real mode in kvm/qemu

[Paolo Bonzini]

On 26/09/19 10:59, Maxim Levitsky wrote:
> If you mean to ask if there is a way to let guest access use no
> paging at all, that is access host physical addresses directly, then
> indeed there is no way, since regular non 'unrestricted guest' mode
> required both protected mode and paging, and 'unrestricted guest'
> requires EPT. Academically speaking it is of course possible to
> create paging tables that are 1:1...

Not so academically, it's exactly what KVM does.  However, indeed it
would also be possible to switch out of EPT mode when CR0.PG=0.  I'm not
sure why it was done this way, maybe when the code was written it was
simpler to use the identity map.

Let's see if Avi is listening... :)

Paolo

Re: Questions about the real mode in kvm/qemu

[Maxim Levitsky]

On Thu, 2019-09-26 at 11:18 +0200, Paolo Bonzini wrote:
> On 26/09/19 10:59, Maxim Levitsky wrote:
> > If you mean to ask if there is a way to let guest access use no
> > paging at all, that is access host physical addresses directly, then
> > indeed there is no way, since regular non 'unrestricted guest' mode
> > required both protected mode and paging, and 'unrestricted guest'
> > requires EPT. Academically speaking it is of course possible to
> > create paging tables that are 1:1...
> 
> Not so academically, it's exactly what KVM does.
You mean KVM uses 1:1 EPT pages and no guest paging,
to allow guest to access host physical address space?
That would break the security completely, thus I think you
mean something else here.


>   However, indeed it
> would also be possible to switch out of EPT mode when CR0.PG=0.  I'm not
> sure why it was done this way, maybe when the code was written it was
> simpler to use the identity map.
> 
> Let's see if Avi is listening... :)
> 
> Paolo

Here a quote from the PRM:

"The first processors to support VMX operation require CR0.PE and CR0.PG to be 1 in VMX operation (see Section
23.8). This restriction implies that guest software cannot be run in unpaged protected mode or in real-address
mode. Later processors support a VM-execution control called “unrestricted guest”. 1 If this control is 1, CR0.PE and
CR0.PG may be 0 in VMX non-root operation. Such processors allow guest software to run in unpaged protected
mode or in real-address mode. The following items describe the behavior of such software:"
...

"As noted in Section 26.2.1.1, the “enable EPT” VM-execution control must be 1 if the “unrestricted guest” VM-execution control is 1."


Best regards,
	Maxim Levitsky

Re: Questions about the real mode in kvm/qemu

[Paolo Bonzini]

On 26/09/19 11:24, Maxim Levitsky wrote:
> On Thu, 2019-09-26 at 11:18 +0200, Paolo Bonzini wrote:
>> On 26/09/19 10:59, Maxim Levitsky wrote:
>>> If you mean to ask if there is a way to let guest access use no
>>> paging at all, that is access host physical addresses directly, then
>>> indeed there is no way, since regular non 'unrestricted guest' mode
>>> required both protected mode and paging, and 'unrestricted guest'
>>> requires EPT. Academically speaking it is of course possible to
>>> create paging tables that are 1:1...
>>
>> Not so academically, it's exactly what KVM does.
> You mean KVM uses 1:1 EPT pages and no guest paging,
> to allow guest to access host physical address space?

No, it uses the usual HVA->GPA EPT pages and 1:1 GPA->GVA pages when EPT
is enabled and guest CR0.PG=0.  This lets KVM work around the CR0.PG=1
requirement when unrestricted guest mode.

Thinking more about it, I suppose that saves memory (the same EPT page
tables can now be used independent of guest CR0.PG), at the cost of
making TLB misses a little slower.

Thanks,

Paolo

>>   However, indeed it
>> would also be possible to switch out of EPT mode when CR0.PG=0.  I'm not
>> sure why it was done this way, maybe when the code was written it was
>> simpler to use the identity map.
>>
>> Let's see if Avi is listening... :)
>>
>> Paolo
> 
> Here a quote from the PRM:
> 
> "The first processors to support VMX operation require CR0.PE and CR0.PG to be 1 in VMX operation (see Section
> 23.8). This restriction implies that guest software cannot be run in unpaged protected mode or in real-address
> mode. Later processors support a VM-execution control called “unrestricted guest”. 1 If this control is 1, CR0.PE and
> CR0.PG may be 0 in VMX non-root operation. Such processors allow guest software to run in unpaged protected
> mode or in real-address mode. The following items describe the behavior of such software:"
> ...
> 
> "As noted in Section 26.2.1.1, the “enable EPT” VM-execution control must be 1 if the “unrestricted guest” VM-execution control is 1."
> 
> 
> Best regards,
> 	Maxim Levitsky
> 

Re: Questions about the real mode in kvm/qemu

[Maxim Levitsky]

On Thu, 2019-09-26 at 11:15 +0200, Paolo Bonzini wrote:
> On 26/09/19 09:52, Li Qiang wrote:
> > Hi Paolo and all,
> > 
> > There are some question about the emulation for real mode in kvm/qemu.
> > For all the 
> > question I suppose the 'unstrict guest' is not enabled. 
> > 
> > 1. how the protected mode CPU emulate the real mode? It seems it uses
> > vm86, however, vm86 is not available in x86_64 CPU? So what's the
> > 'to_vmx(vcpu)->rmode.vm86_active' here vm86 means?
> 
> vm86 mode is available in 64-bit CPUs; it is not available in 64-bit
> mode (EFER.LMA=1).
> 
> Once KVM places the processor in VMX non-root mode (VMLAUNCH/VMRESUME),
> the processor can leave 64-bit mode and run in vm86 mode.  And that's
> exactly what KVM does on processors without unrestricted guest support.
> 
> In that mode, KVM will start trapping all exceptions and send them to
> handle_rmode_exception.  This in turn will either emulate privileged
> instructions (for #GP or #SS) or inject them as realmode exceptions.
Ah, so you run vm86 inside the guest, to emulate the real mode.
That is clever :-)

> 
> However...
> 
> > 2. Does the guest's real mode code run directly in native CPU? It seems
> > 'vmx->emulation_required' is also be false, it the vmx_vcpu_run will do
> > a switch to guest.
> 
> ... as soon as the guest tries to enter protected mode, it will get into
> a situation which is not real mode but doesn't have the segment
> registers properly loaded with selectors.  Therefore, it will either
> hack things together (enter_pmode) or emulate instructions until the
> state is accepted even without unrestricted guest support.
> 
> The "hacking things together" part however does not work for big real
> mode (where you enter 32-bit mode, load segment registers with a flat 4G
> segment, and go back to real mode with the 4G segments).  In big real
> mode, therefore, KVM cannot use vm86 and will keep emulating (slowly -
> up to 1000x slower than unrestricted guest).
That is also understandable.

> 
> > 3. How the EPT work in guest real mode? The EPT is for GVA->GPA->HPA,
> > however there is no GVA, seems the identity mapping does something. But
> > there also some confusion for me. For example the real mode uses CS*4 +
> > IP to address the code.  Who does this calculation? In the kernel emulator? 
> 
> Right, and in fact the CR0.PG and CR0.PE bits must be 1 when running in
> VMX non-root mode with unrestricted guest disabled.
> 
> Therefore, KVM places a 4 KiB identity map at an address provided by
> userspace (with KVM_SET_IDENTITY_MAP_ADDR).  When 1) EPT is enabled 2)
> unrestricted guest isn't 3) CR0.PG=0, KVM points CR3 to it and runs the
> guest with CR0.PG=1, CR4.PAE=0, CR4.PSE=1.  This CR4 setup enables 4 MiB
> huge pages so that a 4 GiB identity map fits in 4 KiB.  This is the only
> case where EPT is enabled but CR3 loads and stores will trap to the
> hypervisor.  This way, the guest CR3 value is faked in the vmexit
> handler, but the processor always uses the identity GVA->GPA mapping.

Ah, so you fake the guest's paging table, not the EPT table. That makes
sense.

Thankfully all of this is not needed when unrestricted guest is available.

Best regards,
	Maxim Levitsky

Re: Questions about the real mode in kvm/qemu

[Li Qiang]

[-- Attachment #1: Type: text/plain, Size: 3572 bytes --]

Paolo Bonzini <pbonzini@redhat.com> 于2019年9月26日周四 下午5:15写道：

> On 26/09/19 09:52, Li Qiang wrote:
> > Hi Paolo and all,
> >
> > There are some question about the emulation for real mode in kvm/qemu.
> > For all the
> > question I suppose the 'unstrict guest' is not enabled.
> >
> > 1. how the protected mode CPU emulate the real mode? It seems it uses
> > vm86, however, vm86 is not available in x86_64 CPU? So what's the
> > 'to_vmx(vcpu)->rmode.vm86_active' here vm86 means?
>
> vm86 mode is available in 64-bit CPUs; it is not available in 64-bit
> mode (EFER.LMA=1).
>
> Once KVM places the processor in VMX non-root mode (VMLAUNCH/VMRESUME),
> the processor can leave 64-bit mode and run in vm86 mode.  And that's
> exactly what KVM does on processors without unrestricted guest support.
>
> In that mode, KVM will start trapping all exceptions and send them to
> handle_rmode_exception.  This in turn will either emulate privileged
> instructions (for #GP or #SS) or inject them as realmode exceptions.
>
>

So without unrestrict guest the mainline is this:
KVM set guest's rflag bit X86_EFLAGS_VM, so when the guest enter guest
mode, it is in vm86 mode.
In this mode, the CPU will access the address like in real
mode(seg*4+offset), this address is linear address. And in fact, the vm86
is still
in protected, so the linear address will be translated to gpa by the
identity mapping table. Then goes to EPT table?

Is this understanding right?



> However...
>
> > 2. Does the guest's real mode code run directly in native CPU? It seems
> > 'vmx->emulation_required' is also be false, it the vmx_vcpu_run will do
> > a switch to guest.
>
> ... as soon as the guest tries to enter protected mode, it will get into
> a situation which is not real mode but doesn't have the segment
> registers properly loaded with selectors.

Therefore, it will either
> hack things together (enter_pmode) or emulate instructions until the
> state is accepted even without unrestricted guest support.
>

Could you please explain this situation more detailed? Why this happen?

Thanks,
Li Qiang



>
> The "hacking things together" part however does not work for big real
> mode (where you enter 32-bit mode, load segment registers with a flat 4G
> segment, and go back to real mode with the 4G segments).  In big real
> mode, therefore, KVM cannot use vm86 and will keep emulating (slowly -
> up to 1000x slower than unrestricted guest).
>
> > 3. How the EPT work in guest real mode? The EPT is for GVA->GPA->HPA,
> > however there is no GVA, seems the identity mapping does something. But
> > there also some confusion for me. For example the real mode uses CS*4 +
> > IP to address the code.  Who does this calculation? In the kernel
> emulator?
>
> Right, and in fact the CR0.PG and CR0.PE bits must be 1 when running in
> VMX non-root mode with unrestricted guest disabled.
>
> Therefore, KVM places a 4 KiB identity map at an address provided by
> userspace (with KVM_SET_IDENTITY_MAP_ADDR).  When 1) EPT is enabled 2)
> unrestricted guest isn't 3) CR0.PG=0, KVM points CR3 to it and runs the
> guest with CR0.PG=1, CR4.PAE=0, CR4.PSE=1.  This CR4 setup enables 4 MiB
> huge pages so that a 4 GiB identity map fits in 4 KiB.  This is the only
> case where EPT is enabled but CR3 loads and stores will trap to the
> hypervisor.  This way, the guest CR3 value is faked in the vmexit
> handler, but the processor always uses the identity GVA->GPA mapping.
>
> Paolo
>
>

[-- Attachment #2: Type: text/html, Size: 4963 bytes --]

Re: Questions about the real mode in kvm/qemu

[Maxim Levitsky]

On Thu, 2019-09-26 at 11:33 +0200, Paolo Bonzini wrote:
> On 26/09/19 11:24, Maxim Levitsky wrote:
> > On Thu, 2019-09-26 at 11:18 +0200, Paolo Bonzini wrote:
> > > On 26/09/19 10:59, Maxim Levitsky wrote:
> > > > If you mean to ask if there is a way to let guest access use no
> > > > paging at all, that is access host physical addresses directly, then
> > > > indeed there is no way, since regular non 'unrestricted guest' mode
> > > > required both protected mode and paging, and 'unrestricted guest'
> > > > requires EPT. Academically speaking it is of course possible to
> > > > create paging tables that are 1:1...
> > > 
> > > Not so academically, it's exactly what KVM does.
> > 
> > You mean KVM uses 1:1 EPT pages and no guest paging,
> > to allow guest to access host physical address space?
> 
> No, it uses the usual HVA->GPA EPT pages and 1:1 GPA->GVA pages when EPT
> is enabled and guest CR0.PG=0.  This lets KVM work around the CR0.PG=1
> requirement when unrestricted guest mode.
I understand now.

> 
> Thinking more about it, I suppose that saves memory (the same EPT page
> tables can now be used independent of guest CR0.PG), at the cost of
> making TLB misses a little slower.
Don't really understand what you mean. 
Isn't this always the case that EPT and guest paging
are independent (at least when no nesting is involved)?


Best regards,
	Maxim Levitsky

Re: Questions about the real mode in kvm/qemu

[Paolo Bonzini]

On 26/09/19 11:35, Li Qiang wrote:
> So without unrestrict guest the mainline is this: KVM set guest's
> rflag bit X86_EFLAGS_VM, so when the guest enter guest mode, it is in
> vm86 mode. In this mode, the CPU will access the address like in
> real mode(seg*4+offset), this address is linear address. And in fact,
> the vm86 is still in protected, so the linear address will be
> translated to gpa by the identity mapping table. Then goes to EPT
> table?

Yes.

>     ... as soon as the guest tries to enter protected mode, it will get into
>     a situation which is not real mode but doesn't have the segment
>     registers properly loaded with selectors.  
> 
>     Therefore, it will either
>     hack things together (enter_pmode) or emulate instructions until the
>     state is accepted even without unrestricted guest support.
> 
> Could you please explain this situation more detailed? Why this happen?

Protected mode entry looks like this:

        mov %cr0, %eax
        or $1, %al
        mov %eax, %cr0
	# [1] now in 16-bit protected mode
        lgdtl gdt32
        ljmpl $8, 2f
	# [2] now in 32-bit protected mode
2:
        .code32
        mov $16, %ax
        mov %ax, %ds
        mov %ax, %es
        mov %ax, %fs
        mov %ax, %gs
        mov %ax, %ss
	# [3] now everything is okay

Between [1] and [3] the vmentry could fail if not in unrestricted mode.
 For example (see checks on guest segment registers in the SDM):

- "CS. Type must be 9, 11, 13, or 15 (accessed code segment)."  CS in
real-mode is a RW data segment, not a code segment.  This applies
between [1] and [2].

- "SS. If the guest will not be virtual-8086 and the “unrestricted
guest” VM-execution control is 0, the RPL (bits 1:0) must equal the RPL
of the selector field for CS."  This may not be the case if the segment
register still holds real-mode values (which are not selectors, just
base >> 4).  This applies between [1] and [3].

- "DS, ES, FS, GS. The DPL cannot be less than the RPL in the selector
field"   Again, the real-mode DPL is zero but the RPL makes no sense if
the segment registers hold a real-mode value.

You can find more about these checks in guest_state_valid(); look at the
"else" branch of that function, the "then" branch is for pmode->rmode
transitions.  When any of the checks fail, KVM emulates instructions
instead of using VMX non-root mode (usually it's just a handful of them,
as in the case above).

Paolo

Re: Questions about the real mode in kvm/qemu

[Paolo Bonzini]

On 26/09/19 11:41, Maxim Levitsky wrote:
>> Thinking more about it, I suppose that saves memory (the same EPT page
>> tables can now be used independent of guest CR0.PG), at the cost of
>> making TLB misses a little slower.
> Don't really understand what you mean. 
> Isn't this always the case that EPT and guest paging
> are independent (at least when no nesting is involved)?

There are two possibilities:

1) emulate CR0.PG=0 with EPT + identity page

- advantage: the EPT pages will be reused once the guest sets CR0.PG=1

- disadvantage: TLB misses have to walk two levels of page tables

2) emulate CR0.PG=0 with EPT disabled.  Similar to ept=0, CR3 will point
to PAE page tables that do the HVA->GPA transition.

- advantage: faster TLB misses

- disadvantage: need to build separate page tables for CR0.PG=1 (EPT
format) and CR0.PG=0 (PAE format), need to "waste" 4k of GPA space for
the identity map

Paolo

Re: Questions about the real mode in kvm/qemu

[Maxim Levitsky]

On Thu, 2019-09-26 at 12:00 +0200, Paolo Bonzini wrote:
> On 26/09/19 11:41, Maxim Levitsky wrote:
> > > Thinking more about it, I suppose that saves memory (the same EPT page
> > > tables can now be used independent of guest CR0.PG), at the cost of
> > > making TLB misses a little slower.
> > 
> > Don't really understand what you mean. 
> > Isn't this always the case that EPT and guest paging
> > are independent (at least when no nesting is involved)?
> 
> There are two possibilities:
> 
> 1) emulate CR0.PG=0 with EPT + identity page
> 
> - advantage: the EPT pages will be reused once the guest sets CR0.PG=1
> 
> - disadvantage: TLB misses have to walk two levels of page tables
> 
> 2) emulate CR0.PG=0 with EPT disabled.  Similar to ept=0, CR3 will point
> to PAE page tables that do the HVA->GPA transition.
> 
> - advantage: faster TLB misses
> 
> - disadvantage: need to build separate page tables for CR0.PG=1 (EPT
> format) and CR0.PG=0 (PAE format), need to "waste" 4k of GPA space for
> the identity map
Thanks for the explanation!

Best regards,
	Maxim Levitsky

Re: Questions about the real mode in kvm/qemu

[Li Qiang]

[-- Attachment #1: Type: text/plain, Size: 2860 bytes --]

Paolo Bonzini <pbonzini@redhat.com> 于2019年9月26日周四 下午5:53写道：

> On 26/09/19 11:35, Li Qiang wrote:
> > So without unrestrict guest the mainline is this: KVM set guest's
> > rflag bit X86_EFLAGS_VM, so when the guest enter guest mode, it is in
> > vm86 mode. In this mode, the CPU will access the address like in
> > real mode(seg*4+offset), this address is linear address. And in fact,
> > the vm86 is still in protected, so the linear address will be
> > translated to gpa by the identity mapping table. Then goes to EPT
> > table?
>
> Yes.
>
> >     ... as soon as the guest tries to enter protected mode, it will get
> into
> >     a situation which is not real mode but doesn't have the segment
> >     registers properly loaded with selectors.
> >
> >     Therefore, it will either
> >     hack things together (enter_pmode) or emulate instructions until the
> >     state is accepted even without unrestricted guest support.
> >
> > Could you please explain this situation more detailed? Why this happen?
>
> Protected mode entry looks like this:
>
>         mov %cr0, %eax
>         or $1, %al
>         mov %eax, %cr0
>         # [1] now in 16-bit protected mode
>         lgdtl gdt32
>         ljmpl $8, 2f
>         # [2] now in 32-bit protected mode
> 2:
>         .code32
>         mov $16, %ax
>         mov %ax, %ds
>         mov %ax, %es
>         mov %ax, %fs
>         mov %ax, %gs
>         mov %ax, %ss
>         # [3] now everything is okay
>
> Between [1] and [3] the vmentry could fail if not in unrestricted mode.
>  For example (see checks on guest segment registers in the SDM):
>
> - "CS. Type must be 9, 11, 13, or 15 (accessed code segment)."  CS in
> real-mode is a RW data segment, not a code segment.  This applies
> between [1] and [2].
>
> - "SS. If the guest will not be virtual-8086 and the “unrestricted
> guest” VM-execution control is 0, the RPL (bits 1:0) must equal the RPL
> of the selector field for CS."  This may not be the case if the segment
> register still holds real-mode values (which are not selectors, just
> base >> 4).  This applies between [1] and [3].
>
> - "DS, ES, FS, GS. The DPL cannot be less than the RPL in the selector
> field"   Again, the real-mode DPL is zero but the RPL makes no sense if
> the segment registers hold a real-mode value.
>
> You can find more about these checks in guest_state_valid(); look at the
> "else" branch of that function, the "then" branch is for pmode->rmode
> transitions.  When any of the checks fail, KVM emulates instructions
> instead of using VMX non-root mode (usually it's just a handful of them,
> as in the case above).
>
>
Thanks so much for your explanation. I will read the code more to
strengthen my understanding.

Thanks,
Li Qiang



> Paolo
>
>

[-- Attachment #2: Type: text/html, Size: 3717 bytes --]

Re: Questions about the real mode in kvm/qemu

[Avi Kivity]

On 9/26/19 12:18 PM, Paolo Bonzini wrote:
> On 26/09/19 10:59, Maxim Levitsky wrote:
>> If you mean to ask if there is a way to let guest access use no
>> paging at all, that is access host physical addresses directly, then
>> indeed there is no way, since regular non 'unrestricted guest' mode
>> required both protected mode and paging, and 'unrestricted guest'
>> requires EPT. Academically speaking it is of course possible to
>> create paging tables that are 1:1...
> Not so academically, it's exactly what KVM does.  However, indeed it
> would also be possible to switch out of EPT mode when CR0.PG=0.  I'm not
> sure why it was done this way, maybe when the code was written it was
> simpler to use the identity map.
>
> Let's see if Avi is listening... :)


I think it was just simpler for the people who implemented it at the 
time. Switching out of EPT would have been a better solution as it would 
no longer require allocating guest physical address space with the few 
warts that requires.

Re: Questions about the real mode in kvm/qemu

[Li Qiang]

[-- Attachment #1: Type: text/plain, Size: 1085 bytes --]

Paolo Bonzini <pbonzini@redhat.com> 于2019年9月26日周四 下午5:18写道：

> On 26/09/19 10:59, Maxim Levitsky wrote:
> > If you mean to ask if there is a way to let guest access use no
> > paging at all, that is access host physical addresses directly, then
> > indeed there is no way, since regular non 'unrestricted guest' mode
> > required both protected mode and paging, and 'unrestricted guest'
> > requires EPT. Academically speaking it is of course possible to
> > create paging tables that are 1:1...
>
> Not so academically, it's exactly what KVM does.  However, indeed it
> would also be possible to switch out of EPT mode when CR0.PG=0.  I'm not
> sure why it was done this way, maybe when the code was written it was
> simpler to use the identity map.
>


Hi Paolo, what's the meaning of 'switch out of EPT mode'. Do you mean when
the guest in real mode emulation(vm86)
can do something to disable EPT? I don't find the code. Seems my
understanding is wrong.

Thanks,
Li Qiang



>
> Let's see if Avi is listening... :)
>
> Paolo
>

[-- Attachment #2: Type: text/html, Size: 1772 bytes --]