[Qemu-devel] i386: EFER vs 32-bit CPU

[Qemu-devel] i386: EFER vs 32-bit CPU

[Pavel Dovgalyuk]

Hello!

 

I found this while debugging the inconsistent saved/restored state of the virtual machine.

 

i386 (32 bit) emulation uses this register (in wrmsr and in MMU fault processing).

But it does not included in VMState, because "efer" field is disabled with #ifdef TARGET_X86_64

 

I think that efer should be saved for i386 too, but how it should be made to preserve the backward
compatibility?

Should we add a section for i386, which duplicates efer, or just version_id of the main VMSD should
be updated?

 

Pavel Dovgalyuk

 

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Dr. David Alan Gilbert]

* Pavel Dovgalyuk (dovgaluk@ispras.ru) wrote:
> Hello!
> 
>  
> 
> I found this while debugging the inconsistent saved/restored state of the virtual machine.
> 
>  
> 
> i386 (32 bit) emulation uses this register (in wrmsr and in MMU fault processing).
> 
> But it does not included in VMState, because "efer" field is disabled with #ifdef TARGET_X86_64
> 
>  
> 
> I think that efer should be saved for i386 too, but how it should be made to preserve the backward
> compatibility?
> 
> Should we add a section for i386, which duplicates efer, or just version_id of the main VMSD should
> be updated?

You could do:

#ifdef TARGET_X86_64
        VMSTATE_UINT64(env.efer, X86CPU),
        VMSTATE_UINT64(env.star, X86CPU),
...

to become:

        VMSTATE_UINT64_TEST(env.efer, X86CPU, efer_needed)
#ifdef TARGET_X86_64
        VMSTATE_UINT64(env.star, X86CPU),


and then make efer_needed a function that returns true if
TARGET_X86_64 or it's a new machine type that knows about whatever
you're going to do with it.

Dave

>  
> 
> Pavel Dovgalyuk
> 
>  
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Paolo Bonzini]

On 29/05/19 13:30, Dr. David Alan Gilbert wrote:
>> Should we add a section for i386, which duplicates efer, or just version_id of the main VMSD should
>> be updated?
> 
> You could do:
> 
> #ifdef TARGET_X86_64
>         VMSTATE_UINT64(env.efer, X86CPU),
>         VMSTATE_UINT64(env.star, X86CPU),
> ...
> 
> to become:
> 
>         VMSTATE_UINT64_TEST(env.efer, X86CPU, efer_needed)
> #ifdef TARGET_X86_64
>         VMSTATE_UINT64(env.star, X86CPU),
> 
> 
> and then make efer_needed a function that returns true if
> TARGET_X86_64 or it's a new machine type that knows about whatever
> you're going to do with it.

I prefer adding a subsection for 32-bit, so that EFER is saved/restored
if nonzero.

Paolo

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Pavel Dovgalyuk]

> From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> On 29/05/19 13:30, Dr. David Alan Gilbert wrote:
> >> Should we add a section for i386, which duplicates efer, or just version_id of the main
> VMSD should
> >> be updated?
> >
> > You could do:
> >
> > #ifdef TARGET_X86_64
> >         VMSTATE_UINT64(env.efer, X86CPU),
> >         VMSTATE_UINT64(env.star, X86CPU),
> > ...
> >
> > to become:
> >
> >         VMSTATE_UINT64_TEST(env.efer, X86CPU, efer_needed)
> > #ifdef TARGET_X86_64
> >         VMSTATE_UINT64(env.star, X86CPU),
> >
> >
> > and then make efer_needed a function that returns true if
> > TARGET_X86_64 or it's a new machine type that knows about whatever
> > you're going to do with it.
> 
> I prefer adding a subsection for 32-bit, so that EFER is saved/restored
> if nonzero.

The question is: should we remove EFER from the original VMSD?

Pavel Dovgalyuk

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Dr. David Alan Gilbert]

* Pavel Dovgalyuk (dovgaluk@ispras.ru) wrote:
> > From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> > On 29/05/19 13:30, Dr. David Alan Gilbert wrote:
> > >> Should we add a section for i386, which duplicates efer, or just version_id of the main
> > VMSD should
> > >> be updated?
> > >
> > > You could do:
> > >
> > > #ifdef TARGET_X86_64
> > >         VMSTATE_UINT64(env.efer, X86CPU),
> > >         VMSTATE_UINT64(env.star, X86CPU),
> > > ...
> > >
> > > to become:
> > >
> > >         VMSTATE_UINT64_TEST(env.efer, X86CPU, efer_needed)
> > > #ifdef TARGET_X86_64
> > >         VMSTATE_UINT64(env.star, X86CPU),
> > >
> > >
> > > and then make efer_needed a function that returns true if
> > > TARGET_X86_64 or it's a new machine type that knows about whatever
> > > you're going to do with it.
> > 
> > I prefer adding a subsection for 32-bit, so that EFER is saved/restored
> > if nonzero.
> 
> The question is: should we remove EFER from the original VMSD?

No, because that would break 64bit compat.

Dave

> Pavel Dovgalyuk
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Pavel Dovgalyuk]

> From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com]
> * Pavel Dovgalyuk (dovgaluk@ispras.ru) wrote:
> > > From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> > > On 29/05/19 13:30, Dr. David Alan Gilbert wrote:
> > > >> Should we add a section for i386, which duplicates efer, or just version_id of the main
> > > VMSD should
> > > >> be updated?
> > > >
> > > > You could do:
> > > >
> > > > #ifdef TARGET_X86_64
> > > >         VMSTATE_UINT64(env.efer, X86CPU),
> > > >         VMSTATE_UINT64(env.star, X86CPU),
> > > > ...
> > > >
> > > > to become:
> > > >
> > > >         VMSTATE_UINT64_TEST(env.efer, X86CPU, efer_needed)
> > > > #ifdef TARGET_X86_64
> > > >         VMSTATE_UINT64(env.star, X86CPU),
> > > >
> > > >
> > > > and then make efer_needed a function that returns true if
> > > > TARGET_X86_64 or it's a new machine type that knows about whatever
> > > > you're going to do with it.
> > >
> > > I prefer adding a subsection for 32-bit, so that EFER is saved/restored
> > > if nonzero.
> >
> > The question is: should we remove EFER from the original VMSD?
> 
> No, because that would break 64bit compat.

Thanks. Already posted a patch here:
https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06796.html

Pavel Dovgalyuk

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Peter Xu]

On Wed, May 29, 2019 at 02:26:39PM +0300, Pavel Dovgalyuk wrote:
> Hello!
> 
>  
> 
> I found this while debugging the inconsistent saved/restored state of the virtual machine.
> 
>  
> 
> i386 (32 bit) emulation uses this register (in wrmsr and in MMU fault processing).

Sorry if this question is elementary, but... why would a 32bit guest
use IA32_EFER?  From SDM I only see 4 bits defined in this MSR (SCE,
LME, LMA, NXE) but is there any of them that should be set in a 32bit
guest?

Regards,

-- 
Peter Xu

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Pavel Dovgalyuk]

> From: Peter Xu [mailto:peterx@redhat.com]
> On Wed, May 29, 2019 at 02:26:39PM +0300, Pavel Dovgalyuk wrote:
> > Hello!
> >
> >
> >
> > I found this while debugging the inconsistent saved/restored state of the virtual machine.
> >
> >
> >
> > i386 (32 bit) emulation uses this register (in wrmsr and in MMU fault processing).
> 
> Sorry if this question is elementary, but... why would a 32bit guest
> use IA32_EFER?  From SDM I only see 4 bits defined in this MSR (SCE,
> LME, LMA, NXE) but is there any of them that should be set in a 32bit
> guest?

Ubuntu server 16.04 (32 bit) sets NXE while booting.
NXE affects the MMU fault processing and exception generation.

Pavel Dovgalyuk

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Peter Xu]

On Thu, May 30, 2019 at 08:54:38AM +0300, Pavel Dovgalyuk wrote:
> > From: Peter Xu [mailto:peterx@redhat.com]
> > On Wed, May 29, 2019 at 02:26:39PM +0300, Pavel Dovgalyuk wrote:
> > > Hello!
> > >
> > >
> > >
> > > I found this while debugging the inconsistent saved/restored state of the virtual machine.
> > >
> > >
> > >
> > > i386 (32 bit) emulation uses this register (in wrmsr and in MMU fault processing).
> > 
> > Sorry if this question is elementary, but... why would a 32bit guest
> > use IA32_EFER?  From SDM I only see 4 bits defined in this MSR (SCE,
> > LME, LMA, NXE) but is there any of them that should be set in a 32bit
> > guest?
> 
> Ubuntu server 16.04 (32 bit) sets NXE while booting.
> NXE affects the MMU fault processing and exception generation.

But this is what I read from the spec (SDM 4.6.1):

Instruction fetches:

  - Access rights depend on the mode of the linear address, the paging
    mode, and the value of IA32_EFER.NXE:

    - For 32-bit paging or if IA32_EFER.NXE = 0, instructions may be
      fetched from any user-mode address.

    - For PAE paging or 4-level paging with IA32_EFER.NXE = 1,
      instructions may be fetched from any user-mode address with a
      translation for which the XD flag is 0 in every paging-structure
      entry controlling the translation.

    - Instructions may not be fetched from any supervisor-mode address.

I'm not an expert of x86 arch but it seems to me that no matter what
NXE bit should be meaningless on x86 32bit according to above.

Also, above spec seems to match with the kvm code too, since in
init_kvm_tdp_mmu() where kvm_mmu.nx is only set with either long mode
or PAE, but never 32bit.  So I'm a bit confused on why that should be
migrated for 32bit (or even, whether should EFER MSR be visible to
such a guest at all?).

Regards,

-- 
Peter Xu

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[Peter Xu]

On Thu, May 30, 2019 at 05:00:28PM +0800, Peter Xu wrote:
> On Thu, May 30, 2019 at 08:54:38AM +0300, Pavel Dovgalyuk wrote:
> > > From: Peter Xu [mailto:peterx@redhat.com]
> > > On Wed, May 29, 2019 at 02:26:39PM +0300, Pavel Dovgalyuk wrote:
> > > > Hello!
> > > >
> > > >
> > > >
> > > > I found this while debugging the inconsistent saved/restored state of the virtual machine.
> > > >
> > > >
> > > >
> > > > i386 (32 bit) emulation uses this register (in wrmsr and in MMU fault processing).
> > > 
> > > Sorry if this question is elementary, but... why would a 32bit guest
> > > use IA32_EFER?  From SDM I only see 4 bits defined in this MSR (SCE,
> > > LME, LMA, NXE) but is there any of them that should be set in a 32bit
> > > guest?
> > 
> > Ubuntu server 16.04 (32 bit) sets NXE while booting.
> > NXE affects the MMU fault processing and exception generation.
> 
> But this is what I read from the spec (SDM 4.6.1):
> 
> Instruction fetches:
> 
>   - Access rights depend on the mode of the linear address, the paging
>     mode, and the value of IA32_EFER.NXE:
> 
>     - For 32-bit paging or if IA32_EFER.NXE = 0, instructions may be
>       fetched from any user-mode address.
> 
>     - For PAE paging or 4-level paging with IA32_EFER.NXE = 1,
>       instructions may be fetched from any user-mode address with a
>       translation for which the XD flag is 0 in every paging-structure
>       entry controlling the translation.
> 
>     - Instructions may not be fetched from any supervisor-mode address.
> 
> I'm not an expert of x86 arch but it seems to me that no matter what
> NXE bit should be meaningless on x86 32bit according to above.
> 
> Also, above spec seems to match with the kvm code too, since in
> init_kvm_tdp_mmu() where kvm_mmu.nx is only set with either long mode
> or PAE, but never 32bit.  So I'm a bit confused on why that should be
> migrated for 32bit (or even, whether should EFER MSR be visible to
> such a guest at all?).

Ouch, I obviously overlooked on that PAE is 32bit... I am sorry for
the noise.

-- 
Peter Xu

Re: [Qemu-devel] i386: EFER vs 32-bit CPU

[TeLeMan]

On Wed, May 29, 2019 at 7:27 PM Pavel Dovgalyuk <dovgaluk@ispras.ru> wrote:
>
> Hello!
>
>
>
> I found this while debugging the inconsistent saved/restored state of the virtual machine.
>
>
>
> i386 (32 bit) emulation uses this register (in wrmsr and in MMU fault processing).
>
> But it does not included in VMState, because "efer" field is disabled with #ifdef TARGET_X86_64
>
>
>
> I think that efer should be saved for i386 too, but how it should be made to preserve the backward
> compatibility?
>
> Should we add a section for i386, which duplicates efer, or just version_id of the main VMSD should
> be updated?
>
>
>
> Pavel Dovgalyuk
>
>
>

Nobody followed this issue 10 years ago:

https://lists.gnu.org/archive/html/qemu-devel/2009-10/msg01765.html