From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42711) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z02Wj-0000RY-Rl for qemu-devel@nongnu.org; Wed, 03 Jun 2015 02:51:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z02Wi-0002tk-Nb for qemu-devel@nongnu.org; Wed, 03 Jun 2015 02:51:49 -0400 Received: from mail-ob0-x235.google.com ([2607:f8b0:4003:c01::235]:34341) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z02Wi-0002tW-IG for qemu-devel@nongnu.org; Wed, 03 Jun 2015 02:51:48 -0400 Received: by obew15 with SMTP id w15so648572obe.1 for ; Tue, 02 Jun 2015 23:51:47 -0700 (PDT) MIME-Version: 1.0 Date: Wed, 3 Jun 2015 14:51:47 +0800 Message-ID: From: Sandhya Kumar Content-Type: multipart/alternative; boundary=001a113ddd2e26d1030517977e9d Subject: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org --001a113ddd2e26d1030517977e9d Content-Type: text/plain; charset=UTF-8 [Query on intended logic] I am trying to learn qemu's MMU emulation logic for x86 and came across H. Peter Anvin's SMAP commit (link ). I have the following doubt on the intended logic (apologies if it is trivial) As per my understanding (which matches versions prior to this commit), we generally maintain only two TLBs [one for kernel and one for user] in x86 ISA for caching address translations. With this commit we seem to have three modes of MMU, although only two will be actually used (either KSMAP or KNOSMAP). Is my claim valid ? Why cannot those two original modes serve the purpose and why is the separation (of KNOMAP and KSMAP) needed? Thanks in advance, Sandhya --001a113ddd2e26d1030517977e9d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
[Query on intended logic]

I = am trying to learn qemu's MMU emulation logic for x86 and came across H= . Peter Anvin's SMAP commit (link). I have the following doubt = on the intended logic (apologies if it is trivial)

As per my understanding (which matches versions prior to this commit), we = generally maintain only two TLBs [one for kernel and one for user] in x86 I= SA for caching address translations. With this commit we seem to have three= modes of MMU, although only two will be actually used (either KSMAP or KNO= SMAP). Is my claim valid ? Why cannot those two original modes serve the pu= rpose and why is the separation (of KNOMAP and KSMAP) needed?
Thanks in advance,
Sandhya
--001a113ddd2e26d1030517977e9d-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45224) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z02gE-0007AC-Mq for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:01:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z02g9-0000jN-OK for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:01:38 -0400 Received: from mail-wi0-x22e.google.com ([2a00:1450:400c:c05::22e]:33850) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z02g9-0000iJ-Hg for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:01:33 -0400 Received: by wibut5 with SMTP id ut5so91765197wib.1 for ; Wed, 03 Jun 2015 00:01:32 -0700 (PDT) Sender: Paolo Bonzini Message-ID: <556EA649.3010805@redhat.com> Date: Wed, 03 Jun 2015 09:01:29 +0200 From: Paolo Bonzini MIME-Version: 1.0 References: In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Sandhya Kumar , qemu-devel@nongnu.org On 03/06/2015 08:51, Sandhya Kumar wrote: > As per my understanding (which matches versions prior to this commit), > we generally maintain only two TLBs [one for kernel and one for user] in > x86 ISA for caching address translations. With this commit we seem to > have three modes of MMU, although only two will be actually used (either > KSMAP or KNOSMAP). This is not accurate. If AC=0, data accesses from the kernel use KNOSMAP, but implicit accesses (e.g. reads of the IDT) use KSMAP. > Is my claim valid ? Why cannot those two original > modes serve the purpose and why is the separation (of KNOMAP and KSMAP) > needed? Because the QEMU TLB just has a single bit for "is this page readable". In supervisor mode and with SMAP enabled, this changes depending on the value of the AC bit. Without separate TLBs for KNOSMAP/KSMAP, you would have to flush the TLB on every CLAC or STAC instruction. Paolo From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53630) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03IW-0008Rk-Td for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:41:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z03IV-00085i-JF for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:41:12 -0400 Received: from mail-oi0-x236.google.com ([2607:f8b0:4003:c06::236]:36204) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03IV-00085c-Em for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:41:11 -0400 Received: by oihb142 with SMTP id b142so1266116oih.3 for ; Wed, 03 Jun 2015 00:41:10 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <556EA649.3010805@redhat.com> References: <556EA649.3010805@redhat.com> Date: Wed, 3 Jun 2015 15:41:10 +0800 Message-ID: From: Sandhya Kumar Content-Type: multipart/alternative; boundary=001a113ce070c85ffc0517982e6a Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: qemu-devel@nongnu.org --001a113ce070c85ffc0517982e6a Content-Type: text/plain; charset=UTF-8 Thanks for your mail. Are these TLB modes logic specific to QEMU implementation for x86? Asking this as I am not able to get any information about seperate TLBs from Intel developer manuals On Wed, Jun 3, 2015 at 3:01 PM, Paolo Bonzini wrote: > > > On 03/06/2015 08:51, Sandhya Kumar wrote: > > As per my understanding (which matches versions prior to this commit), > > we generally maintain only two TLBs [one for kernel and one for user] in > > x86 ISA for caching address translations. With this commit we seem to > > have three modes of MMU, although only two will be actually used (either > > KSMAP or KNOSMAP). > > This is not accurate. If AC=0, data accesses from the kernel use > KNOSMAP, but implicit accesses (e.g. reads of the IDT) use KSMAP. > > > Is my claim valid ? Why cannot those two original > > modes serve the purpose and why is the separation (of KNOMAP and KSMAP) > > needed? > > Because the QEMU TLB just has a single bit for "is this page readable". > In supervisor mode and with SMAP enabled, this changes depending on the > value of the AC bit. Without separate TLBs for KNOSMAP/KSMAP, you would > have to flush the TLB on every CLAC or STAC instruction. > > Paolo > --001a113ce070c85ffc0517982e6a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for your mail.=C2=A0 Are these TLB modes logic spec= ific to QEMU implementation for x86?=C2=A0
Asking this as I am not able= to get any information about seperate TLBs from Intel developer manuals

On Wed, = Jun 3, 2015 at 3:01 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:


On 03/06/2015 08:51, Sandhya Kumar wrote:
> As per my understanding (which matches versions prior to this commit),=
> we generally maintain only two TLBs [one for kernel and one for user] = in
> x86 ISA for caching address translations. With this commit we seem to<= br> > have three modes of MMU, although only two will be actually used (eith= er
> KSMAP or KNOSMAP).

This is not accurate.=C2=A0 If AC=3D0, data accesses from the kernel= use
KNOSMAP, but implicit accesses (e.g. reads of the IDT) use KSMAP.

> Is my claim valid ? Why cannot those two original
> modes serve the purpose and why is the separation (of KNOMAP and KSMAP= )
> needed?

Because the QEMU TLB just has a single bit for "is this page re= adable".
=C2=A0In supervisor mode and with SMAP enabled, this changes depending on t= he
value of the AC bit.=C2=A0 Without separate TLBs for KNOSMAP/KSMAP, you wou= ld
have to flush the TLB on every CLAC or STAC instruction.

Paolo

--001a113ce070c85ffc0517982e6a-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56661) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03Zf-0008Ts-II for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:58:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z03Zc-0005Yw-C0 for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:58:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55754) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03Zc-0005Yc-60 for qemu-devel@nongnu.org; Wed, 03 Jun 2015 03:58:52 -0400 Message-ID: <556EB3B8.2020402@redhat.com> Date: Wed, 03 Jun 2015 09:58:48 +0200 From: Paolo Bonzini MIME-Version: 1.0 References: <556EA649.3010805@redhat.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Sandhya Kumar Cc: qemu-devel@nongnu.org On 03/06/2015 09:41, Sandhya Kumar wrote: > Thanks for your mail. Are these TLB modes logic specific to QEMU > implementation for x86? Yes, they are specific to QEMU. > Asking this as I am not able to get any information about seperate TLBs > from Intel developer manuals Real hardware TLBs probably work in a completely different (and undocumented) way. Paolo From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59308) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03iP-0002VO-9O for qemu-devel@nongnu.org; Wed, 03 Jun 2015 04:07:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z03iO-0002z1-CP for qemu-devel@nongnu.org; Wed, 03 Jun 2015 04:07:57 -0400 Received: from mail-oi0-x234.google.com ([2607:f8b0:4003:c06::234]:35220) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03iO-0002yv-7R for qemu-devel@nongnu.org; Wed, 03 Jun 2015 04:07:56 -0400 Received: by oihd6 with SMTP id d6so1682804oih.2 for ; Wed, 03 Jun 2015 01:07:55 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <556EB3B8.2020402@redhat.com> References: <556EA649.3010805@redhat.com> <556EB3B8.2020402@redhat.com> Date: Wed, 3 Jun 2015 16:07:55 +0800 Message-ID: From: Sandhya Kumar Content-Type: multipart/alternative; boundary=001a113ddd2e707e650517988e51 Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: qemu-devel@nongnu.org --001a113ddd2e707e650517988e51 Content-Type: text/plain; charset=UTF-8 Thanks again. One more question. On versions prior to the mentioned commit, is there any specific reason (in x86 source code of QEMU) to choose separate modes for address translations (of kernel and user mode)? Or was that done just for performance improvement? On Wed, Jun 3, 2015 at 3:58 PM, Paolo Bonzini wrote: > > > On 03/06/2015 09:41, Sandhya Kumar wrote: > > Thanks for your mail. Are these TLB modes logic specific to QEMU > > implementation for x86? > > Yes, they are specific to QEMU. > > > Asking this as I am not able to get any information about seperate TLBs > > from Intel developer manuals > > Real hardware TLBs probably work in a completely different (and > undocumented) way. > > Paolo > --001a113ddd2e707e650517988e51 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks again. One more question.

On ver= sions prior to the mentioned commit, is there any specific reason (in x86 s= ource code of QEMU) to choose separate modes for address translations (of k= ernel and user mode)? Or was that done just for performance improvement?



On Wed, Jun 3, = 2015 at 3:58 PM, Paolo Bonzini <pbonzini@redhat.com> wrote= :


On 03/06/2015 09:41, Sandhya Kumar wrote:
> Thanks for your mail.=C2=A0 Are these TLB modes logic specific to QEMU=
> implementation for x86?

Yes, they are specific to QEMU.

> Asking this as I am not able to get any information about seperate TLB= s
> from Intel developer manuals

Real hardware TLBs probably work in a completely different (and
undocumented) way.

Paolo

--001a113ddd2e707e650517988e51-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34523) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03wu-0006Rr-EZ for qemu-devel@nongnu.org; Wed, 03 Jun 2015 04:22:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z03wp-00008z-26 for qemu-devel@nongnu.org; Wed, 03 Jun 2015 04:22:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54673) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z03wo-00008u-Sj for qemu-devel@nongnu.org; Wed, 03 Jun 2015 04:22:50 -0400 Message-ID: <556EB956.1080301@redhat.com> Date: Wed, 03 Jun 2015 10:22:46 +0200 From: Paolo Bonzini MIME-Version: 1.0 References: <556EA649.3010805@redhat.com> <556EB3B8.2020402@redhat.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Sandhya Kumar Cc: qemu-devel@nongnu.org On 03/06/2015 10:07, Sandhya Kumar wrote: > Thanks again. One more question. > > On versions prior to the mentioned commit, is there any specific reason > (in x86 source code of QEMU) to choose separate modes for address > translations (of kernel and user mode)? Or was that done just for > performance improvement? It's because pages can be accessible only from kernel space, so the protection bits for the pages can be different in the two TLBs. Paolo From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52048) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z04uz-0002JE-4A for qemu-devel@nongnu.org; Wed, 03 Jun 2015 05:25:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z04uy-0002kU-3B for qemu-devel@nongnu.org; Wed, 03 Jun 2015 05:25:01 -0400 Received: from mail-ob0-x230.google.com ([2607:f8b0:4003:c01::230]:35992) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z04ux-0002kJ-UF for qemu-devel@nongnu.org; Wed, 03 Jun 2015 05:25:00 -0400 Received: by objn8 with SMTP id n8so3030251obj.3 for ; Wed, 03 Jun 2015 02:24:59 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <556EB956.1080301@redhat.com> References: <556EA649.3010805@redhat.com> <556EB3B8.2020402@redhat.com> <556EB956.1080301@redhat.com> Date: Wed, 3 Jun 2015 17:24:59 +0800 Message-ID: From: Sandhya Kumar Content-Type: multipart/alternative; boundary=089e0153761208d587051799a21f Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: qemu-devel@nongnu.org --089e0153761208d587051799a21f Content-Type: text/plain; charset=UTF-8 Well, I think we can also achieve this like adding a flag in the structure of CPUTLBEntry. Am I missing something? On Wed, Jun 3, 2015 at 4:22 PM, Paolo Bonzini wrote: > > > On 03/06/2015 10:07, Sandhya Kumar wrote: > > Thanks again. One more question. > > > > On versions prior to the mentioned commit, is there any specific reason > > (in x86 source code of QEMU) to choose separate modes for address > > translations (of kernel and user mode)? Or was that done just for > > performance improvement? > > It's because pages can be accessible only from kernel space, so the > protection bits for the pages can be different in the two TLBs. > > Paolo > --089e0153761208d587051799a21f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Well, I think we can also achieve this like adding a flag = in the structure of CPUTLBEntry.=C2=A0
Am I missing something?


O= n Wed, Jun 3, 2015 at 4:22 PM, Paolo Bonzini <pbonzini@redhat.com&g= t; wrote:


On 03/06/2015 10:07, Sandhya Kumar wrote:
> Thanks again. One more question.
>
> On versions prior to the mentioned commit, is there any specific reaso= n
> (in x86 source code of QEMU) to choose separate modes for address
> translations (of kernel and user mode)? Or was that done just for
> performance improvement?

It's because pages can be accessible only from kernel space, so = the
protection bits for the pages can be different in the two TLBs.

Paolo

--089e0153761208d587051799a21f-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56021) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z056I-0004i6-Qh for qemu-devel@nongnu.org; Wed, 03 Jun 2015 05:36:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z056F-00023D-J5 for qemu-devel@nongnu.org; Wed, 03 Jun 2015 05:36:42 -0400 Received: from mail-ig0-f178.google.com ([209.85.213.178]:34100) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z056F-000231-8E for qemu-devel@nongnu.org; Wed, 03 Jun 2015 05:36:39 -0400 Received: by igbhj9 with SMTP id hj9so107352251igb.1 for ; Wed, 03 Jun 2015 02:36:38 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <556EA649.3010805@redhat.com> <556EB3B8.2020402@redhat.com> <556EB956.1080301@redhat.com> From: Peter Maydell Date: Wed, 3 Jun 2015 10:36:18 +0100 Message-ID: Content-Type: text/plain; charset=UTF-8 Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Sandhya Kumar Cc: Paolo Bonzini , QEMU Developers On 3 June 2015 at 10:24, Sandhya Kumar wrote: > Well, I think we can also achieve this like adding a flag in the structure > of CPUTLBEntry. > Am I missing something? The point of the TLB data structure is to allow very fast access in the common case of "TLB hit to guest RAM". If we had extra flags to check in this code path it would slow it down. At the moment the code only needs to look up the entry in the TLB for the mmu_index it wants, and if it finds a hit it knows that it is valid. -- PMM From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41299) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z18eM-0004iz-Nv for qemu-devel@nongnu.org; Sat, 06 Jun 2015 03:36:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z18eL-00056f-F5 for qemu-devel@nongnu.org; Sat, 06 Jun 2015 03:36:14 -0400 Received: from mail-ob0-x231.google.com ([2607:f8b0:4003:c01::231]:35083) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z18eL-00056R-8K for qemu-devel@nongnu.org; Sat, 06 Jun 2015 03:36:13 -0400 Received: by obbgp2 with SMTP id gp2so46668418obb.2 for ; Sat, 06 Jun 2015 00:36:12 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <556EA649.3010805@redhat.com> <556EB3B8.2020402@redhat.com> <556EB956.1080301@redhat.com> Date: Sat, 6 Jun 2015 15:36:12 +0800 Message-ID: From: Sandhya Kumar Content-Type: multipart/alternative; boundary=001a11c32e068436ee0517d476c1 Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: peter.maydell@linaro.org Cc: Paolo Bonzini , qemu-devel@nongnu.org --001a11c32e068436ee0517d476c1 Content-Type: text/plain; charset=UTF-8 Thanks Peter for your explanation. [The following question on TLB working could be a deviation from the first mail here, but asking here instead of starting new thread.] I picked up a simple 'Hello world' ELF executable (shown at the end) and tried to experiment with QEMU's address translations (i.e. guest VA -> host VA in *softmmu_template.h*) occurring in userland for that process. This is the sequence of guest VA (in hexadecimal) being translated: *401bee* *401c07* *401c0e* *401c13* 401d23 401d39 402009 ...... and so on The *italized* ones (first four) belong to *_start* of my executable and the next few can be traced to *__libc_start_main *in my executable*. *Can anyone please help me understand why the order is appearing like this? Intuitively, I would guess to be as in every instruction of executable (401bee, 401bf0 etc). Also there was no context switch during the above which rules out TLB getting flushed at random time points. Let me know if you need more information. ./hello: file format elf64-x86-64 Disassembly of section .text: 0000000000401bee <_start>: 401bee: 31 ed xor %ebp,%ebp 401bf0: 49 89 d1 mov %rdx,%r9 401bf3: 5e pop %rsi 401bf4: 48 89 e2 mov %rsp,%rdx 401bf7: 48 83 e4 f0 and $0xfffffffffffffff0,%rsp ...... and so on On Wed, Jun 3, 2015 at 5:36 PM, Peter Maydell wrote: > On 3 June 2015 at 10:24, Sandhya Kumar > wrote: > > Well, I think we can also achieve this like adding a flag in the > structure > > of CPUTLBEntry. > > Am I missing something? > > The point of the TLB data structure is to allow very fast access > in the common case of "TLB hit to guest RAM". If we had extra > flags to check in this code path it would slow it down. At the > moment the code only needs to look up the entry in the TLB > for the mmu_index it wants, and if it finds a hit it knows that > it is valid. > > -- PMM > --001a11c32e068436ee0517d476c1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Peter for your explanation.

[The= following question on TLB working could be a deviation from the first mail= here, but asking here instead of starting new thread.]

I picked up a simple 'Hello world' ELF executable (shown at t= he end) and tried to experiment with QEMU's address translations (i.e. = guest VA -> host VA in softmmu_template.h) occurring in userland = for that process. This is the sequence of guest VA (in hexadecimal) being t= ranslated:

401bee
401c07
401c0e
401c13
401d23
401d39
402009 =C2=A0
...... and so on
<= div>
The italized ones (first four) belong to _star= t=C2=A0of my executable and the next few can be traced to __libc_sta= rt_main in my executable. Can anyone please help me understand w= hy the order is appearing like this?=C2=A0 Intuitively, I would guess to be= as in every instruction of executable (401bee, 401bf0 etc). Also there was= no context switch during the above which rules out TLB getting flushed at = random time points. Let me know if you need more information.

./hello: =C2=A0 =C2=A0 file format elf64-x8= 6-64

Disassembly of section .text:
<= br>
0000000000401bee <_start>:
=C2=A0 401bee:= =C2=A0 =C2=A0 =C2=A0 31 ed =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 xor =C2=A0 =C2=A0%ebp,%ebp
=C2=A0 401bf0: = =C2=A0 =C2=A0 =C2=A0 49 89 d1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0mov =C2=A0 =C2=A0%rdx,%r9
=C2=A0 401bf3: =C2=A0 =C2=A0 = =C2=A0 5e =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0pop =C2=A0 =C2=A0%rsi
=C2=A0 401bf4: =C2=A0 =C2=A0 = =C2=A0 48 89 e2 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mov = =C2=A0 =C2=A0%rsp,%rdx
=C2=A0 401bf7: =C2=A0 =C2=A0 =C2=A0 48 83 = e4 f0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 and =C2=A0 =C2=A0$0xfffffff= ffffffff0,%rsp

...... and so on


On Wed, Jun 3, 2015 at 5:36 PM, Peter Maydell <peter.ma= ydell@linaro.org> wrote:
On 3 June 2015 at 10:24, Sandhya Kumar <insatiablecuriousity07@gmail.com>= wrote:
> Well, I think we can also achieve this like adding a flag in the struc= ture
> of CPUTLBEntry.
> Am I missing something?

The point of the TLB data structure is to allow very fast access
in the common case of "TLB hit to guest RAM". If we had extra
flags to check in this code path it would slow it down. At the
moment the code only needs to look up the entry in the TLB
for the mmu_index it wants, and if it finds a hit it knows that
it is valid.

-- PMM

--001a11c32e068436ee0517d476c1-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39645) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z1Mfa-00006o-4V for qemu-devel@nongnu.org; Sat, 06 Jun 2015 18:34:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z1MfV-0007DV-94 for qemu-devel@nongnu.org; Sat, 06 Jun 2015 18:34:26 -0400 Received: from mail-ie0-f173.google.com ([209.85.223.173]:32973) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z1MfV-0007D6-4n for qemu-devel@nongnu.org; Sat, 06 Jun 2015 18:34:21 -0400 Received: by iebgx4 with SMTP id gx4so77750128ieb.0 for ; Sat, 06 Jun 2015 15:34:20 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <556EA649.3010805@redhat.com> <556EB3B8.2020402@redhat.com> <556EB956.1080301@redhat.com> From: Peter Maydell Date: Sat, 6 Jun 2015 23:34:00 +0100 Message-ID: Content-Type: text/plain; charset=UTF-8 Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Sandhya Kumar Cc: Paolo Bonzini , QEMU Developers On 6 June 2015 at 08:36, Sandhya Kumar wrote: > Thanks Peter for your explanation. > > [The following question on TLB working could be a deviation from the first > mail here, but asking here instead of starting new thread.] > > I picked up a simple 'Hello world' ELF executable (shown at the end) and > tried to experiment with QEMU's address translations (i.e. guest VA -> host > VA in softmmu_template.h) occurring in userland for that process. This is > the sequence of guest VA (in hexadecimal) being translated: > > 401bee > 401c07 > 401c0e > 401c13 > 401d23 > 401d39 > 402009 > ...... and so on > > The italized ones (first four) belong to _start of my executable and the > next few can be traced to __libc_start_main in my executable. Can anyone > please help me understand why the order is appearing like this? Most code loads don't go through the softmmu_template.h code. The frontend (target-*/translate.c) calls cpu_ld*_code functions, which are implemented by macros in include/exec/cpu_ldst_template.h. Those functions will try to do a direct lookup in QEMU's TLB first, and will only call the helper functions in softmmu_template.h if they miss. So you're not going to see a call for every instruction. (My guess is you're seeing one call every basic block, but it's not possible to tell from the detail you give.) -- PMM From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46834) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z1nAI-0000gI-M2 for qemu-devel@nongnu.org; Sun, 07 Jun 2015 22:51:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z1nAH-0003SX-39 for qemu-devel@nongnu.org; Sun, 07 Jun 2015 22:51:54 -0400 Received: from mail-oi0-x22a.google.com ([2607:f8b0:4003:c06::22a]:36771) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z1nAG-0003SR-RO for qemu-devel@nongnu.org; Sun, 07 Jun 2015 22:51:52 -0400 Received: by oihb142 with SMTP id b142so84697189oih.3 for ; Sun, 07 Jun 2015 19:51:51 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <556EA649.3010805@redhat.com> <556EB3B8.2020402@redhat.com> <556EB956.1080301@redhat.com> Date: Mon, 8 Jun 2015 10:51:51 +0800 Message-ID: From: Sandhya Kumar Content-Type: multipart/alternative; boundary=047d7b4722b04e831e0517f8b947 Subject: Re: [Qemu-devel] On x86 MMU modes List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Maydell , Paolo Bonzini , qemu-devel@nongnu.org --047d7b4722b04e831e0517f8b947 Content-Type: text/plain; charset=UTF-8 Thanks Peter for your response. I notice that *tlb_fill()* is happening only in *softmmu_template.h *and not anywhere else in code base. This means I should expect the TLB itself to be populated here for other code loads to have successful look up later. Am I wrong with my understanding? Even I guessed TLB to be fetching basic block (i.e. chunk of code with single entry and exit), but realized its not and hence I posted the question here. If we look at the sequence mentioned earlier in thread (i.e. 401bee , 401c07, 401c0e, 401c13) 401c07 and 401c0e forms a pattern - it is from the immediate value after "mov" opcode, modRM bytes in lines 10, 11. I also checked other "mov" lines. This pattern appears to match everywhere *expect* for the "mov" in line 9 . Let me know if you need more information. [My executable] 0000000000401bee <_start>: 401bee: 31 ed xor %ebp,%ebp 401bf0: 49 89 d1 mov %rdx,%r9 401bf3: 5e pop %rsi 401bf4: 48 89 e2 mov %rsp,%rdx 401bf7: 48 83 e4 f0 and $0xfffffffffffffff0,%rsp 401bfb: 50 push %rax 401bfc: 54 push %rsp 401bfd: 49 c7 c0 20 24 40 00 mov $0x402420,%r8 // [Line 9] 401c04: 48 c7 c1 90 23 40 00 mov $0x402390,%rcx // [Line 10] 401c0b: 48 c7 c7 fe 1c 40 00 mov $0x401cfe,%rdi // [Line 11] 401c12: e8 09 01 00 00 callq 401d20 <__libc_start_main> 401c17: f4 hlt 401c18: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1) 401c1f: 00 On Sun, Jun 7, 2015 at 6:34 AM, Peter Maydell wrote: > On 6 June 2015 at 08:36, Sandhya Kumar > wrote: > > Thanks Peter for your explanation. > > > > [The following question on TLB working could be a deviation from the > first > > mail here, but asking here instead of starting new thread.] > > > > I picked up a simple 'Hello world' ELF executable (shown at the end) and > > tried to experiment with QEMU's address translations (i.e. guest VA -> > host > > VA in softmmu_template.h) occurring in userland for that process. This is > > the sequence of guest VA (in hexadecimal) being translated: > > > > 401bee > > 401c07 > > 401c0e > > 401c13 > > 401d23 > > 401d39 > > 402009 > > ...... and so on > > > > The italized ones (first four) belong to _start of my executable and the > > next few can be traced to __libc_start_main in my executable. Can anyone > > please help me understand why the order is appearing like this? > > Most code loads don't go through the softmmu_template.h code. The > frontend (target-*/translate.c) calls cpu_ld*_code functions, which > are implemented by macros in include/exec/cpu_ldst_template.h. Those > functions will try to do a direct lookup in QEMU's TLB first, and will > only call the helper functions in softmmu_template.h if they miss. > So you're not going to see a call for every instruction. (My guess is > you're seeing one call every basic block, but it's not possible to tell > from the detail you give.) > > -- PMM > --047d7b4722b04e831e0517f8b947 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Peter for your response. I notice that=C2=A0tlb_fill()=C2=A0 is happening only in softmmu_template.h and no= t anywhere else in code base. This means I should expect the TLB itself to = be populated here for other code loads to have successful look up later. Am= I wrong with my understanding?

Even I guessed TLB= to be fetching basic block (i.e. chunk of code with single entry and exit)= , but realized its not and hence I posted the question here. If we look at = the sequence mentioned earlier in thread (i.e. 401bee , 401c07, 401c0e, 401= c13)
401c07 and 401c0e forms a pattern - it is from the immediate= value after "mov" opcode, modRM bytes in lines 10, 11. I also ch= ecked other "mov" lines. This pattern appears to match everywhere= expect for the "mov" in line 9 .

Let me know if you need more information.

[M= y executable]

0000000000401bee <_sta= rt>:
=C2=A0 401bee: =C2=A0 =C2=A0 =C2=A0 31 ed =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 xor =C2=A0 =C2=A0%ebp,= %ebp
=C2=A0 401bf0: =C2=A0 =C2=A0 =C2=A0 49 89 d1 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mov =C2=A0 =C2=A0%rdx,%r9
=C2=A0 401bf3: =C2=A0 =C2=A0 =C2=A0 5e =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pop =C2=A0 =C2=A0%rsi
= =C2=A0 401bf4: =C2=A0 =C2=A0 =C2=A0 48 89 e2 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0mov =C2=A0 =C2=A0%rsp,%rdx
=C2=A0 401b= f7: =C2=A0 =C2=A0 =C2=A0 48 83 e4 f0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 and =C2=A0 =C2=A0$0xfffffffffffffff0,%rsp
=C2=A0 401bfb: =C2= =A0 =C2=A0 =C2=A0 50 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0push =C2=A0 %rax
=C2=A0 401bfc: =C2=A0 = =C2=A0 =C2=A0 54 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0push =C2=A0 %rsp
=C2=A0 401bfd: =C2=A0 =C2=A0= =C2=A0 49 c7 c0 20 24 40 00 =C2=A0 =C2=A0mov =C2=A0 =C2=A0$0x402420,%r8 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 // [Line 9]
=C2=A0 401c04: =C2=A0 =C2=A0 =C2=A0 48 c7 c1 90 23 40 00 =C2=A0 =C2=A0mo= v =C2=A0 =C2=A0$0x402390,%rcx =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 // [Line 10]
=C2=A0 401c0b: =C2=A0 =C2=A0 =C2=A0 48 c7 c7 fe = 1c 40 00 =C2=A0 =C2=A0mov =C2=A0 =C2=A0$0x401cfe,%rdi =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// [Line 11]
=C2=A0 401c= 12: =C2=A0 =C2=A0 =C2=A0 e8 09 01 00 00 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0c= allq =C2=A0401d20 <__libc_start_main>
=C2=A0 401c17: =C2=A0= =C2=A0 =C2=A0 f4 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0hlt =C2=A0 =C2=A0
=C2=A0 401c18: =C2=A0 =C2= =A0 =C2=A0 0f 1f 84 00 00 00 00 =C2=A0 =C2=A0nopl =C2=A0 0x0(%rax,%rax,1)
=C2=A0 401c1f: =C2=A0 =C2=A0 =C2=A0 00

<= div>

On Sun, Jun 7, 2015 at 6:34 AM, Peter Maydell <peter.maydell@lin= aro.org> wrote:
On 6 June 2015 at 08:36, Sandhya Kumar <insatiablecuriousity07@gmail.com> wrote: > Thanks Peter for your explanation.
>
> [The following question on TLB working could be a deviation from the f= irst
> mail here, but asking here instead of starting new thread.]
>
> I picked up a simple 'Hello world' ELF executable (shown at th= e end) and
> tried to experiment with QEMU's address translations (i.e. guest V= A -> host
> VA in softmmu_template.h) occurring in userland for that process. This= is
> the sequence of guest VA (in hexadecimal) being translated:
>
> 401bee
> 401c07
> 401c0e
> 401c13
> 401d23
> 401d39
> 402009
> ...... and so on
>
> The italized ones (first four) belong to _start of my executable and t= he
> next few can be traced to __libc_start_main in my executable. Can anyo= ne
> please help me understand why the order is appearing like this?

Most code loads don't go through the softmmu_template.h code. Th= e
frontend (target-*/translate.c) calls cpu_ld*_code functions, which
are implemented by macros in include/exec/cpu_ldst_template.h. Those
functions will try to do a direct lookup in QEMU's TLB first, and will<= br> only call the helper functions in softmmu_template.h if they miss.
So you're not going to see a call for every instruction. (My guess is you're seeing one call every basic block, but it's not possible to = tell
from the detail you give.)

-- PMM

--047d7b4722b04e831e0517f8b947--