From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751343AbdKTWpJ (ORCPT ); Mon, 20 Nov 2017 17:45:09 -0500 Received: from mail.efficios.com ([167.114.142.141]:48170 "EHLO mail.efficios.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751149AbdKTWpH (ORCPT ); Mon, 20 Nov 2017 17:45:07 -0500 Date: Mon, 20 Nov 2017 22:46:03 +0000 (UTC) From: Mathieu Desnoyers To: Andi Kleen Cc: Thomas Gleixner , Peter Zijlstra , "Paul E. McKenney" , Boqun Feng , Andy Lutomirski , Dave Watson , linux-kernel , linux-api , Paul Turner , Andrew Morton , Russell King , Ingo Molnar , "H. Peter Anvin" , Andrew Hunter , Chris Lameter , Ben Maurer , rostedt , Josh Triplett , Linus Torvalds , Catalin Marinas , Will Deacon , Michael Kerrisk Message-ID: <1920107276.18703.1511217963487.JavaMail.zimbra@efficios.com> In-Reply-To: <20171120184927.GK2482@two.firstfloor.org> References: <20171114200414.2188-1-mathieu.desnoyers@efficios.com> <20171114200414.2188-9-mathieu.desnoyers@efficios.com> <1766414702.18278.1511194398489.JavaMail.zimbra@efficios.com> <204285712.18480.1511203151076.JavaMail.zimbra@efficios.com> <20171120184927.GK2482@two.firstfloor.org> Subject: Re: [RFC PATCH v3 for 4.15 08/24] Provide cpu_opv system call MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [167.114.142.141] X-Mailer: Zimbra 8.7.11_GA_1854 (ZimbraWebClient - FF52 (Linux)/8.7.11_GA_1854) Thread-Topic: Provide cpu_opv system call Thread-Index: oqg4NfG5lCgn7PKqcGW0f7zGqBLvbA== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org ----- On Nov 20, 2017, at 1:49 PM, Andi Kleen andi@firstfloor.org wrote: >> Having cpu_opv do a 4k memcpy allow it to handle scenarios where >> rseq fails to progress. > > If anybody ever gets that right. It will be really hard to just > test such a path. > > It also seems fairly theoretical to me. Do you even have a > test case where the normal path stops making forward progress? We expect the following loop to progress, typically after a single iteration: do { cpu = rseq_cpu_start(); ret = rseq_addv(&v, 1, cpu); attempts++; } while (ret); Now runnig this in gdb, break on "main", run, and single-step execution with "next", the program is stuck in an infinite loop. What solution do you have in mind to handle this kind of scenario without breaking pre-existing debuggers ? Looking at vDSO examples of vgetcpu and vclock_gettime under gdb 7.7.1 (debian) with glibc 2.19: sched_getcpu behavior under single-stepping per source line with "step" seems to only see the ../sysdeps/unix/sysv/linux/x86_64/sched_getcpu.S source lines, which makes it skip single-stepping of the vDSO. sched_getcpu under "stepi": it does go through the vDSO instruction addresses. It does progress, given that there is no loop there. clock_gettime under "step": it only sees source lines of ../sysdeps/unix/clock_gettime.c. clock_gettime under "stepi": it's stuck in an infinite loop. So instruction-level stepping from gdb turns clock_gettime vDSO into a never-ending loop, which is already bad. But with rseq, the situation is even worse, because it turns source line level single-stepping into infinite loops. My understanding from https://sourceware.org/bugzilla/show_bug.cgi?id=14466 is that GDB currently simply removes the vDSO from its list of library mappings, which is probably why it skips over vDSO for the source lines single-stepping case. We cannot do that with rseq, because we _want_ the rseq critical section to be inlined into the application or library. A function call costs more than most rseq critical sections. I plan to have the rseq user-space code provide a "__rseq_table" section so debuggers can eventually figure out that they need to skip over the rseq critical sections. However, it won't help the fact that pre-existing debugger single-stepping will start turning perfectly working programs into never-ending loops simply by having glibc use rseq for memory allocation. Using the cpu_opv system call on rseq failure solves this problem entirely. I would even go further and recommend to take a similar approach when lack of progress is detected in a vDSO, and invoke the equivalent system call. The current implementation of the clock_gettime() vDSO turns instruction-level single-stepping into never ending loops, which is far from being elegant. Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com