Re: [PATCH 1/2] kvm-unit-tests: Add a func to run instruction in emulator

[Gleb Natapov <gleb@redhat.com>]

On Mon, Jun 10, 2013 at 01:28:48AM +0800, 李春奇 <Arthur Chunqi Li> wrote:
> I have trouble to send the address of inregs into alt_insn_page, so I
> use r9 and r10 to carry %rsp and %rbp into it. And r8 is used to
> trigger vmexit.
> 
If you going to use call you will need stack. I said how not to use it
and how not to use register for trapping instruction since you cannot
use r[0-7] registers because we will obviously want them to be part of
inregs/outreg.

> I paste the relevant functions as follows:
> 
> 
> static void trap_emulator(uint64_t *mem, uint8_t *insn_page,
>     uint8_t *alt_insn_page, void *insn_ram,
>     uint8_t *alt_insn, int alt_insn_length)
> {
>     ulong *cr3 = (ulong *)read_cr3();
>     int i;
>     static struct regs save;
> 
>     // Pad with RET instructions
>     memset(insn_page, 0x90, 4096);
>     memset(alt_insn_page, 0x90, 4096);
> 
>     // Place a trapping instruction in the page to trigger a VMEXIT
>     insn_page[0] = 0x49; //xchg   %rsp, %r9
>     insn_page[1] = 0x87;
>     insn_page[2] = 0xe1;
>     insn_page[3] = 0x49; //xchg   %rbp, %r10
>     insn_page[4] = 0x87;
>     insn_page[5] = 0xea;
>     insn_page[6] = 0x41; // mov %eax, (%r8)
>     insn_page[7] = 0x89;
>     insn_page[8] = 0x00; // ret
> 
>     // Place the instruction we want the hypervisor to see in the alternate page
>     for (i=0; i<alt_insn_length; i++)
>         alt_insn_page[i+6] = alt_insn[i];
>     i+=6;
>     insn_page[i++] = 0x49; //xchg   %rsp, %r9
>     insn_page[i++] = 0x87;
>     insn_page[i++] = 0xe1;
>     insn_page[i++] = 0x49; //xchg   %rbp, %r10
>     insn_page[i++] = 0x87;
>     insn_page[i++] = 0xea;
>     insn_page[i++] = 0xc3; // ret
>     save = inregs;
>     // Load the code TLB with insn_page, but point the page tables at
>     // alt_insn_page (and keep the data TLB clear, for AMD decode assist).
>     // This will make the CPU trap on the insn_page instruction but the
>     // hypervisor will see alt_insn_page.
>     install_page(cr3, virt_to_phys(insn_page), insn_ram);
>     invlpg(insn_ram);
>     // Load code TLB
>     asm volatile("call *%0" : : "r"(insn_ram));
>     install_page(cr3, virt_to_phys(alt_insn_page), insn_ram);
>     // Trap, let hypervisor emulate at alt_insn_page
>     asm volatile(
>     "mov %2, %%r8\n\r"
>     "push 72+%[save]; popf\n\t"
>     "xchg %%rax, 0+%[save]\n\t"
>     "xchg %%rbx, 8+%[save] \n\t"
>     "xchg %%rcx, 16+%[save] \n\t"
>     "xchg %%rdx, 24+%[save] \n\t"
>     "xchg %%rsi, 32+%[save] \n\t"
>     "xchg %%rdi, 40+%[save] \n\t"
>     "xchg %%r9, 48+%[save] \n\t" // %rsp in %r9
>     "xchg %%r10, 56+%[save] \n\t" // %rbp in %r10
> 
>     "call *%1\n\t"
> 
>     "xchg %%rax, 0+%[save] \n\t"
>     "xchg %%rbx, 8+%[save] \n\t"
>     "xchg %%rcx, 16+%[save] \n\t"
>     "xchg %%rdx, 24+%[save] \n\t"
>     "xchg %%rsi, 32+%[save] \n\t"
>     "xchg %%rdi, 40+%[save] \n\t"
>     "xchg %%r9, 48+%[save] \n\t" // %rsp in %r9
>     "xchg %%r10, 56+%[save] \n\t" // %rbp in %r10
>     /* Save RFLAGS in outregs*/
>     "pushf \n\t"
>     "pop 72+%[save] \n\t"
>     : [save]"=m"(save)
>     : "r"(insn_ram), "r"(mem)
>     : "memory", "cc", "r9", "r10", "r8"
>     );
>     outregs = save;
> }
> 
> 
> static void test_mmx_movq_mf(uint64_t *mem, uint8_t *insn_page,
>     uint8_t *alt_insn_page, void *insn_ram)
> {
>     uint16_t fcw = 0;  // all exceptions unmasked
>     uint8_t alt_insn[] = {0x0f, 0x7f, 0x00}; // movq %mm0, (%rax)
> 
>     write_cr0(read_cr0() & ~6);  // TS, EM
>     exceptions = 0;
>     handle_exception(MF_VECTOR, advance_rip_by_3_and_note_exception);
>     asm volatile("fninit; fldcw %0" : : "m"(fcw));
>     asm volatile("fldz; fldz; fdivp"); // generate exception
> 
>     inregs = (struct regs){ .rsp=0, .rbp=0 };
>     trap_emulator(mem, insn_page, alt_insn_page, insn_ram,
> alt_insn, 3);
>     // exit MMX mode
>     asm volatile("fnclex; emms");
>     report("movq mmx generates #MF2", exceptions == 1);
>     handle_exception(MF_VECTOR, 0);
> }
> 
> On Mon, Jun 10, 2013 at 1:13 AM, Gleb Natapov <gleb@redhat.com> wrote:
> > On Mon, Jun 10, 2013 at 01:09:15AM +0800, 李春奇 <Arthur Chunqi Li> wrote:
> >> I have finished the infrastructure but after changing test_mmx_movq_mf
> >> test case, it return error:
> >> unhandled excecption 6
> >> Return value from qemu: 15
> >>
> >> If I don't change %rsp and %rbp, it runs OK.
> >> So I wonder if this test case is strictly reply on %rsp and %rbp?
> >>
> > I can't help without seeing the code.
> >
> >> On Mon, Jun 10, 2013 at 12:00 AM, Gleb Natapov <gleb@redhat.com> wrote:
> >> > On Sun, Jun 09, 2013 at 11:23:26PM +0800, 李春奇 <Arthur Chunqi Li> wrote:
> >> >> On Sun, Jun 9, 2013 at 10:09 PM, Gleb Natapov <gleb@redhat.com> wrote:
> >> >> > On Sun, Jun 09, 2013 at 09:22:27PM +0800, 李春奇 <Arthur Chunqi Li> wrote:
> >> >> >> On Sun, Jun 9, 2013 at 8:49 PM, Gleb Natapov <gleb@redhat.com> wrote:
> >> >> >> > On Sun, Jun 09, 2013 at 08:44:32PM +0800, 李春奇 <Arthur Chunqi Li> wrote:
> >> >> >> >> On Sun, Jun 9, 2013 at 7:07 PM, Gleb Natapov <gleb@redhat.com> wrote:
> >> >> >> >> > On Fri, Jun 07, 2013 at 10:31:38AM +0800, Arthur Chunqi Li wrote:
> >> >> >> >> >> Add a function trap_emulator to run an instruction in emulator.
> >> >> >> >> >> Set inregs first (%rax, %rsp, %rbp, %rflags have special usage and
> >> >> >> >> >> cannot set in inregs), put instruction codec in alt_insn and call
> >> >> >> >> >> func with alt_insn_length. Get results in outregs.
> >> >> >> >> >>
> >> >> >> >> > Why %rax, %rsp, %rbp, %rflags cannot be set in inregs?
> >> >> >> >> >
> >> >> >> >> > %rax because trapping instruction uses it? Use one that does not use
> >> >> >> >> > register at all: MOV r/m32, imm32
> >> >> >> >> I don't know why set %rax before call alt_insn_page can cause error. I
> >> >> >> >> use "xchg %%rax, 0+%[save]" before "call *%1" and the %rcx is not set
> >> >> >> >> correctly.
> >> >> >> > We better find this out :)
> >> >> >> I found that before calling alt_insn_page, address of "mem" is saved
> >> >> >> to %rax, why?
> >> >> > Because instruction that we use to trigger vmexit is mov %eax, (%rax) so
> >> >> > MMOI address mem is loaded into %rax before jumping into it.
> >> >> I think this is why changing %rax will cause error. If we use mov
> >> >> %eax, (%rax) to trigger vmexit, and %rax is changed before calling
> >> >> alt_insn_page, codes in alt_insn_page will not be executed and return
> >> >> directly.
> >> >> I changed the codes which trigger vmexit to "mov %eax, (%r8)" and set
> >> >> "mem" to %r8 before calling alt_insn_page, it runs OK.
> >> >>
> >> > Just use an instruction that does not use registers at all. mov $1, addr
> >> > where addr is immediate and encoded from mem parameter.
> >> >
> >> >> Besides, I also don't know if changed %rflags may cause some
> >> >> unpredictable actions, so now we just treat it with no error :)
> >> > If test sets rflags to a value that causes crashes this is a test bug,
> >> > no need to prevent this from happening.
> >> >
> >> > --
> >> >                         Gleb.
> >>
> >>
> >>
> >> --
> >> Arthur Chunqi Li
> >> Department of Computer Science
> >> School of EECS
> >> Peking University
> >> Beijing, China
> >
> > --
> >                         Gleb.
> 
> 
> 
> -- 
> Arthur Chunqi Li
> Department of Computer Science
> School of EECS
> Peking University
> Beijing, China

--
			Gleb.