From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41511) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bAzRZ-00043Z-V6 for qemu-devel@nongnu.org; Thu, 09 Jun 2016 08:52:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bAzRV-0005ya-17 for qemu-devel@nongnu.org; Thu, 09 Jun 2016 08:52:17 -0400 Received: from mail-lf0-x242.google.com ([2a00:1450:4010:c07::242]:33846) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bAzRU-0005yL-4a for qemu-devel@nongnu.org; Thu, 09 Jun 2016 08:52:12 -0400 Received: by mail-lf0-x242.google.com with SMTP id k192so3394078lfb.1 for ; Thu, 09 Jun 2016 05:52:11 -0700 (PDT) References: <1461073171-22953-1-git-send-email-a.rigo@virtualopensystems.com> <57595637.8030004@gmail.com> From: Sergey Fedorov Message-ID: <57596679.2060208@gmail.com> Date: Thu, 9 Jun 2016 15:52:09 +0300 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC v8 00/14] Slow-path for atomic instruction translation List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: alvise rigo Cc: QEMU Developers , MTTCG Devel , Jani Kokkonen , Claudio Fontana , VirtualOpenSystems Technical Team , =?UTF-8?Q?Alex_Benn=c3=a9e?= , Paolo Bonzini , Richard Henderson On 09/06/16 15:35, alvise rigo wrote: > On Thu, Jun 9, 2016 at 1:42 PM, Sergey Fedorov wrote: >> On 19/04/16 16:39, Alvise Rigo wrote: >>> The implementation heavily uses the software TLB together with a new >>> bitmap that has been added to the ram_list structure which flags, on a >>> per-CPU basis, all the memory pages that are in the middle of a LoadLink >>> (LL), StoreConditional (SC) operation. Since all these pages can be >>> accessed directly through the fast-path and alter a vCPU's linked value, >>> the new bitmap has been coupled with a new TLB flag for the TLB virtual >>> address which forces the slow-path execution for all the accesses to a >>> page containing a linked address. >> But I'm afraid we've got a scalability problem using software TLB engine >> heavily. This approach relies on TLB flush of all CPUs which is not very >> cheap operation. That is going to be even more expansive in case of >> MTTCG as you need to exit the CPU execution loop in order to avoid >> deadlocks. >> >> I see you try mitigate this issue by introducing a history of N last >> pages touched by an exclusive access. That would work fine avoiding >> excessive TLB flushes as long as the current working set of exclusively >> accessed pages does not go beyond N. Once we exceed this limit we'll get >> a global TLB flush on most LL operations. I'm afraid we can get dramatic > Indeed, if the guest does a loop of N+1 atomic operations, at each > iteration we will have N flushes. > >> performance decrease as guest code implements finer-grained locking >> scheme. I would like to emphasise that performance can degrade sharply >> and dramatically as soon as the limit gets exceeded. How could we tackle >> this problem? > In my opinion, the length of the history should not be fixed to avoid > the drawback of above. We can make the history's length dynamic (until > a given threshold is reached) according to the pressure of atomic > instructions. What should remain constant is the time elapsed to make > a cycle of the history's array. We can for instance store in the lower > bits of the addresses in the history a sort of timestamp used to > calculate the period and adjust accordingly the length of the history. > What do you think? It really depends on what algorithm we'll introduce for dynamic history length. I'm afraid it could complicate things and introduce its own overhead. I'm also going to look at Emilio's approach http://thread.gmane.org/gmane.comp.emulators.qemu/335297. Kind regards, Sergey