From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gleb Natapov Subject: Re: [PATCH 1/2] kvm-unit-tests: Add a func to run instruction in emulator Date: Tue, 18 Jun 2013 18:47:54 +0300 Message-ID: <20130618154754.GE21032@redhat.com> References: <1370532278-22063-1-git-send-email-yzt356@gmail.com> <51B8DF0E.3060105@redhat.com> <20130618124517.GA16557@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Paolo Bonzini , kvm To: =?utf-8?B?5j8/5j+l5aU/IDxBcnRodXIgQ2h1bnFpIExpPg==?= Return-path: Received: from mx1.redhat.com ([209.132.183.28]:27937 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755589Ab3FRPr7 convert rfc822-to-8bit (ORCPT ); Tue, 18 Jun 2013 11:47:59 -0400 Content-Disposition: inline In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Tue, Jun 18, 2013 at 10:28:59PM +0800, =E6??=E6?=A5=E5=A5? wrote: > On Tue, Jun 18, 2013 at 8:45 PM, Gleb Natapov wrote= : > > On Thu, Jun 13, 2013 at 05:30:03PM +0800, =E6=9D=8E=E6=98=A5=E5=A5=87= wrote: > >> Hi Gleb, > >> I'm trying to solve these problems in the past days and meet many > >> difficulties. You want to save all the general registers in callin= g > >> insn_page, so registers should be saved to (save) in insn_page. > >> Because all the instructions should be generated outside and copy = to > >> insn_page, and the instructions generated outside is RIP-relative,= so > >> inside insn_page (save) will be wrong pointed with RIP-relative co= de. > >> > > They do not have to be generated outside. You can write code into > > insn_page directly. Something like this outside of any functions: > > > > asm(".align 4096\n\t" > > ".global insn_page\n\t" > > ".global insn_page_end\n\t" > > ".global test_insn\n\t" > > ".global test_insn_end\n\t" > > "insn_page:" > > "mov %%rax, outregs \n\t" > > ... > > "test_insn:\n\t" > > "in (%ds), %al\n\t" > > ". =3D . + 31\n\t" > > "test_insn_end:\n\t" > > "mov outregs, %%rax\n\t" > > ... > > "ret\n\t" > > ".align 4096\n\t" > > "insn_page_end:\n\t"); > > > > Now you copy that into alt_insn_page, put instruction you want to t= est > > into test_insn offset and remap alt_insn_page into "insn_page" virt= ual address. > I used such codes: >=20 > invlpg((void *)virt_to_phys(insn_page)); virt_to_phys? > asm volatile("call *%0" : : "r"(insn_page)); > install_page(cr3, virt_to_phys(alt_insn_page), insn_page); > asm volatile("call *%0": : "r"(insn_page+1)); +1? >=20 > But it seems that alt_insn_page are not remapped to insn_page. Here > insn_page and alt_insn_page are all declared statically with > "asm(...)". >=20 > Arthur > > > >> I have tried to move (save) into insn_page. But when calling > >> insn_page, data in it can only be read and any instructions like "= xchg > >> %%rax, 0+%[save]" may cause error, because at this time read is fr= om > >> TLB but write will cause inconsistent. > >> > >> Another way is disabling RIP-relative code, but I failed when usin= g > >> "-mcmodel-large -fno-pic", the binary is also using RIP-relative m= ode. > >> Is there any way to totally disable RIP-relative code? Besides, us= ing > >> this feature may specified to some newer C compiler. This may not = be a > >> good solution. > >> > >> If we don't set %rsp and %rbp when executing emulator code, we can > >> just use =E2=80=9Cpush/pop" to save other general registers. > >> > >> If you have any better solutions, please let me know. > >> > >> Thanks, > >> Arthur > >> > >> On Thu, Jun 13, 2013 at 12:50 PM, =E6=9D=8E=E6=98=A5=E5=A5=87 > >> wrote: > >> > On Thu, Jun 13, 2013 at 4:50 AM, Paolo Bonzini wrote: > >> >> Il 06/06/2013 11:24, Arthur Chunqi Li ha scritto: > >> >>> Add a function trap_emulator to run an instruction in emulator= =2E > >> >>> Set inregs first (%rax is invalid because it is used as return > >> >>> address), put instruction codec in alt_insn and call func with > >> >>> alt_insn_length. Get results in outregs. > >> >>> > >> >>> Signed-off-by: Arthur Chunqi Li > >> >>> --- > >> >>> x86/emulator.c | 81 +++++++++++++++++++++++++++++++++++++++= +++++++++++++++++ > >> >>> 1 file changed, 81 insertions(+) > >> >>> > >> >>> diff --git a/x86/emulator.c b/x86/emulator.c > >> >>> index 96576e5..8ab9904 100644 > >> >>> --- a/x86/emulator.c > >> >>> +++ b/x86/emulator.c > >> >>> @@ -11,6 +11,14 @@ int fails, tests; > >> >>> > >> >>> static int exceptions; > >> >>> > >> >>> +struct regs { > >> >>> + u64 rax, rbx, rcx, rdx; > >> >>> + u64 rsi, rdi, rsp, rbp; > >> >>> + u64 rip, rflags; > >> >>> +}; > >> >>> + > >> >>> +static struct regs inregs, outregs; > >> >>> + > >> >>> void report(const char *name, int result) > >> >>> { > >> >>> ++tests; > >> >>> @@ -685,6 +693,79 @@ static void test_shld_shrd(u32 *mem) > >> >>> report("shrd (cl)", *mem =3D=3D ((0x12345678 >> 3) | (5u = << 29))); > >> >>> } > >> >>> > >> >>> +static void trap_emulator(uint64_t *mem, uint8_t *insn_page, > >> >>> + uint8_t *alt_insn_page, void *insn_= ram, > >> >>> + uint8_t *alt_insn, int alt_insn_len= gth) > >> >>> +{ > >> >>> + ulong *cr3 =3D (ulong *)read_cr3(); > >> >>> + int i; > >> >>> + > >> >>> + // Pad with RET instructions > >> >>> + memset(insn_page, 0xc3, 4096); > >> >>> + memset(alt_insn_page, 0xc3, 4096); > >> >>> + > >> >>> + // Place a trapping instruction in the page to trigger a= VMEXIT > >> >>> + insn_page[0] =3D 0x89; // mov %eax, (%rax) > >> >>> + insn_page[1] =3D 0x00; > >> >>> + insn_page[2] =3D 0x90; // nop > >> >>> + insn_page[3] =3D 0xc3; // ret > >> >>> + > >> >>> + // Place the instruction we want the hypervisor to see i= n the alternate page > >> >>> + for (i=3D0; i >> >>> + alt_insn_page[i] =3D alt_insn[i]; > >> >>> + > >> >>> + // Save general registers > >> >>> + asm volatile( > >> >>> + "push %rax\n\r" > >> >>> + "push %rbx\n\r" > >> >>> + "push %rcx\n\r" > >> >>> + "push %rdx\n\r" > >> >>> + "push %rsi\n\r" > >> >>> + "push %rdi\n\r" > >> >>> + ); > >> >> > >> >> This will not work if GCC is using rsp-relative addresses to ac= cess > >> >> local variables. You need to use mov instructions to load from= inregs, > >> >> and put the push/pop sequences inside the "main" asm that does = the "call > >> >> *%1". > >> > Is there any way to let gcc use absolute address to access varia= bles? > >> > I move variant "save" to the global and use "xchg %%rax, 0+%[sav= e]" > >> > and it seems that addressing for "save" is wrong. > >> > > >> > Arthur > >> >> > >> >> Paolo > >> >> > >> >>> + // Load the code TLB with insn_page, but point the page = tables at > >> >>> + // alt_insn_page (and keep the data TLB clear, for AMD d= ecode assist). > >> >>> + // This will make the CPU trap on the insn_page instruct= ion but the > >> >>> + // hypervisor will see alt_insn_page. > >> >>> + install_page(cr3, virt_to_phys(insn_page), insn_ram); > >> >>> + invlpg(insn_ram); > >> >>> + // Load code TLB > >> >>> + asm volatile("call *%0" : : "r"(insn_ram + 3)); > >> >>> + install_page(cr3, virt_to_phys(alt_insn_page), insn_ram)= ; > >> >>> + // Trap, let hypervisor emulate at alt_insn_page > >> >>> + asm volatile( > >> >>> + "call *%1\n\r" > >> >>> + > >> >>> + "mov %%rax, 0+%[outregs] \n\t" > >> >>> + "mov %%rbx, 8+%[outregs] \n\t" > >> >>> + "mov %%rcx, 16+%[outregs] \n\t" > >> >>> + "mov %%rdx, 24+%[outregs] \n\t" > >> >>> + "mov %%rsi, 32+%[outregs] \n\t" > >> >>> + "mov %%rdi, 40+%[outregs] \n\t" > >> >>> + "mov %%rsp,48+ %[outregs] \n\t" > >> >>> + "mov %%rbp, 56+%[outregs] \n\t" > >> >>> + > >> >>> + /* Save RFLAGS in outregs*/ > >> >>> + "pushf \n\t" > >> >>> + "popq 72+%[outregs] \n\t" > >> >>> + : [outregs]"+m"(outregs) > >> >>> + : "r"(insn_ram), > >> >>> + "a"(mem), "b"(inregs.rbx), > >> >>> + "c"(inregs.rcx), "d"(inregs.rdx), > >> >>> + "S"(inregs.rsi), "D"(inregs.rdi) > >> >>> + : "memory", "cc" > >> >>> + ); > >> >>> + // Restore general registers > >> >>> + asm volatile( > >> >>> + "pop %rax\n\r" > >> >>> + "pop %rbx\n\r" > >> >>> + "pop %rcx\n\r" > >> >>> + "pop %rdx\n\r" > >> >>> + "pop %rsi\n\r" > >> >>> + "pop %rdi\n\r" > >> >>> + ); > >> >>> +} > >> >>> + > >> >>> static void advance_rip_by_3_and_note_exception(struct ex_reg= s *regs) > >> >>> { > >> >>> ++exceptions; > >> >>> > >> >> > >> > > >> > > >> > > >> > -- > >> > Arthur Chunqi Li > >> > Department of Computer Science > >> > School of EECS > >> > Peking University > >> > Beijing, China > >> > >> > >> > >> -- > >> Arthur Chunqi Li > >> Department of Computer Science > >> School of EECS > >> Peking University > >> Beijing, China > > > > -- > > Gleb. >=20 >=20 >=20 > -- > Arthur Chunqi Li > Department of Computer Science > School of EECS > Peking University > Beijing, China -- Gleb.