From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58086) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VbIUz-0007In-FF for qemu-devel@nongnu.org; Tue, 29 Oct 2013 19:15:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VbIUt-0008Fy-5D for qemu-devel@nongnu.org; Tue, 29 Oct 2013 19:14:57 -0400 Received: from www11.your-server.de ([213.133.104.11]:52006) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VbIUs-0008Fl-SV for qemu-devel@nongnu.org; Tue, 29 Oct 2013 19:14:51 -0400 Message-ID: <52704165.5060406@macke.de> Date: Tue, 29 Oct 2013 16:14:45 -0700 From: Sebastian Macke MIME-Version: 1.0 References: <1383073495-5332-1-git-send-email-sebastian@macke.de> <1383073495-5332-7-git-send-email-sebastian@macke.de> <52702E54.5030801@macke.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 06/13] target-openrisc: Remove TLB flush from l.rfe instruction List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Max Filippov Cc: openrisc@lists.openrisc.net, openrisc@lists.opencores.org, qemu-devel , Ethan Hunt On 29/10/2013 3:20 PM, Max Filippov wrote: > On Wed, Oct 30, 2013 at 1:53 AM, Sebastian Macke wrote: >> On 29/10/2013 2:01 PM, Max Filippov wrote: >>> On Tue, Oct 29, 2013 at 11:04 PM, Sebastian Macke >>> wrote: >>>> At the moment there are two TLBs. The OpenRISC TLB followed >>>> by the QEMU's own TLB. >>>> At the end of the TLB miss handler a tlb_flush of QEMUs TLB >>>> is executed which is exactly what we want to avoid. >>>> As long as there is no context switch we don't have to flush the TLB. >>> So this flush was needed in order to clean QEMU TLB in case >>> DTLB/ITLB translation was enabled/disabled, right? But since you >>> already have mmu index for nommu operation, wouldn't it be easier >>> to indicate mmu index correctly for data and code access and drop >>> this flush? >>> >> The mmu index is already set correctly and this patch removes the flush. > I'm not sure: cpu_mmu_index only checks SR_IME, not SR_DME. And again, correct. I saw this problem too and wanted to correct it later. But using Linux the IME and DME flags are always changed together and therefore distinguishing between them don't play a role. So I forgot it in the end. >> 1. Problem >> The problem is if there is a context switch. OpenRISC clears its own small >> tlb page by page. But this does mean it flushes all pages in the big QEMU > I think there shouldn't be any entries in QEMU TLB other than those in > OpenRISC TLB, otherwise it would be possible to access unmapped virtual > addresses without getting pagefaults. Correct. And this was one dilemma I had. How to increase the whole speed of tlb misses by an order of magnitude and not break any rules. Without the two patches it works like this. User mode OpenRISC TLB-Miss -> Exception -> QEMU TLB flush -> Set new page in the tlb handler -> return via l.rfe -> QEMU TLB flush In the end we had always only one valid page in the QEMU TLB which is kind of crazy. ( Now I am unsure, because with my logic we would have never a valid page in the QEMU TLB and would run in an endless loop) So with the patches QEMU's TLB could have indeed more entries than the OpenRISC TLB right now but which is 99% fine if you run it under Linux. Linux has the option to clear certain pages from the tlb which then of course will not reckognize those pages in QEMUs TLB. So there is a small chance for an error. Three options: 1. Removing QEMUs TLBs. 2. Another option might be to make the flushing mmu_index dependend to increase the speed. But this is not implemented as far as I see. 3. I could also invalidate the previous page if the OpenRISC TLB entry is overwritten. But then I don't know the correct mmu_index. So this has to be done for all possible mmu indices. Option three might be possible. I didn't think about this before. Hopefully OpenRISC will get Hardware TLB refill next year. This would solve the problem. >> TLB. This is the reason why l.rfe instruction clears the tlb which is the >> instruction used to return to user mode >> But according to the specification this is wrong. >> >> 2. Problem which is the case you mentioned. >> Your are right, this is one solution and its written in the patchnotes as >> point 1. >> But this would not solve the problem No 1. I mentioned in this email. >> >> Confused? I am :) >> >> Easy: l.rfe is not supposed to clear the tlb. It can but it shouldn't. >> With this patch I remove the flush and solve all problems by assuming a >> global tlb flush if you invalidate the first entry of the small OpenRISC >> TLB.