From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05F4723E358 for ; Mon, 15 Jun 2026 02:24:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781490282; cv=none; b=NPeS2ryY9jG+wPJCOIBG6S3e8tpuF/XLfFI2otxPOLNvLZ3jXL/AzgRwmeUfUt/w6BNXlRjVq35f84ziLgJnQtgy/VKR5yjHKEpTVUq6eDIlu5uCnTwGdZokf1tDAWJDllzTBk7uipaBFEN8KydRsIdFZds9NtugQYPFYT2TtXQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781490282; c=relaxed/simple; bh=YaqW+JS64OkufL8iiNrtXUYjwdu/BwWlE2Yo6AJRomA=; h=Content-Type:Message-ID:Date:MIME-Version:Subject:From:To:Cc: References:In-Reply-To; b=S0V2zMtg0AQwPS8w1CMo2jYcSS94X0UZtqhl9YXPCs6OdGyrwfh0NqdQV6zFcKLB8YsmkD5nYMT+UcsK0vYDLBlidZcu3zTUYBiUzCfUEOHqtxbHEk4T5cs+v9YaACuHbr1/73ZbfnvhVNlNwi0vXxV1NLu+ToE96FxCpfDbwcY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=esjFM5mV; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="esjFM5mV" Received: from [10.124.221.87] ([192.55.54.47]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 65F27u842706670 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Sun, 14 Jun 2026 19:08:02 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 65F27u842706670 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2026052701; t=1781489283; bh=ilrQnvnxsombF9xQcHvpXUp1C0lQA2PbVnPLJ56BIaE=; h=Date:Subject:From:To:Cc:References:In-Reply-To:From; b=esjFM5mVUsdZ+YulKdtg8nfIITJC+J6mTLEsd/Qkgb4M+sXqYJK7kG9nccRxXe1Ks QJE9tVqlznsiDACFkPHTbTIsHTz5uGhLcPwAWTxUgbVNJVZcSF1p9ePXInRPSmcdDJ cjAB2a0MiVu9P4KFLQ/UBsvAwsBcrltXckUiJBx8ZLqdDZTuwWVOHUytYD/KMj+A8c JCg7J14eh+S2EIO6MFwfLrqckx0sub7882RVQO1rWfNf9yYYSKK+CyfDoBm7R/bBgg ZH52H+4TdmkzjRAFb0c13oIh7Sx+VrWpWHrc28/s2uwRgAsgDi2BhkdQPv65yfnALc hhzUGjUgBDajw== Content-Type: multipart/mixed; boundary="------------QW1Vv0qTW0Zy5HelAHfOJAAK" Message-ID: <01ac45a8-b558-4d4d-9f8f-e7a4e725d5d2@zytor.com> Date: Sun, 14 Jun 2026 19:07:50 -0700 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: 8aeb879baf12 - significant system call latency regression, bisected From: "H. Peter Anvin" To: Peter Zijlstra Cc: tglx@kernel.org, mingo@redhat.com, bp@alien8.de, Nathan Chancellor , Calvin Owens , Dave Hansen , torvalds@linux-foundation.org, x86-ML , LKML References: <20260613085919.GF42921@noisy.programming.kicks-ass.net> <203E61B7-290F-4F87-860F-B352D0072703@zytor.com> <338ead9a-91f4-4579-9954-e18911fa3f68@zytor.com> Content-Language: en-US, sv-SE In-Reply-To: <338ead9a-91f4-4579-9954-e18911fa3f68@zytor.com> This is a multi-part message in MIME format. --------------QW1Vv0qTW0Zy5HelAHfOJAAK Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 2026-06-14 17:19, H. Peter Anvin wrote: > > OK, so v7.1 was released with this sizable performance regression. That > begs the question how to deal with it. > > One option that might be reasonable for -stable is to simply add back 16 > bytes of NOPs into the assembly file. However, that is obviously not a > long term fix. > Okay, here is a hack that actually generates the proper alignment, and it DOES in fact fix the performance regression. It uses the same hack as the Makefile to deal with function alignment with a prefix: it adds unnecessary NOPs so that the pre-alignment and post-alignment are the same. At the end of the day this really ought to be fixed in gcc. This is not meant to be a final patch; this should go in a header file and be cleaned up etc, but I wanted to confirm that it does, in fact, fix the regression and that the alignment of x64_sys_call is the root cause of the problem. PeterZ: at some point you and I talked about the following: - Should x64_sys_call() be noinstr? - If so, any reason we can't inline it into do_syscall_64()? - Since we no longer use the sys_call_table[] as a jump table, do we actually need array_index_nospec()? in do_syscall_x64|32? -hpa --------------QW1Vv0qTW0Zy5HelAHfOJAAK Content-Type: text/plain; charset=UTF-8; name="diff" Content-Disposition: attachment; filename="diff" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL2FyY2gveDg2L2VudHJ5L3N5c2NhbGxfNjQuYyBiL2FyY2gveDg2L2Vu dHJ5L3N5c2NhbGxfNjQuYwppbmRleCA3MWYwMzI1MDRlNzMuLjMzN2UzZTUzZDI2MiAxMDA2 NDQKLS0tIGEvYXJjaC94ODYvZW50cnkvc3lzY2FsbF82NC5jCisrKyBiL2FyY2gveDg2L2Vu dHJ5L3N5c2NhbGxfNjQuYwpAQCAtOSw2ICs5LDE0IEBACiAjaW5jbHVkZSA8bGludXgvbm9z cGVjLmg+CiAjaW5jbHVkZSA8YXNtL3N5c2NhbGwuaD4KIAorI2lmZGVmIENPTkZJR19DQUxM X1BBRERJTkcKKyMgZGVmaW5lIF9wZmUoeCkgX19hdHRyaWJ1dGUoKHBhdGNoYWJsZV9mdW5j dGlvbl9lbnRyeSh4LHgpKSkKKyNlbHNlCisjIGRlZmluZSBfcGZlKHgpCisjZW5kaWYKKyNk ZWZpbmUgX2FsaWduX2Z1bmMoeCkgX19hbGlnbmVkKHgpIF9wZmUoeC1DT05GSUdfRlVOQ1RJ T05fQUxJR05NRU5UK0NPTkZJR19GVU5DVElPTl9QQURESU5HX0JZVEVTKQorI2RlZmluZSBh bGlnbl9mdW5jKHgpIF9hbGlnbl9mdW5jKCh4KSA8IENPTkZJR19GVU5DVElPTl9BTElHTk1F TlQgPyBDT05GSUdfRlVOQ1RJT05fQUxJR05NRU5UIDogKHgpKQorCiAjZGVmaW5lIF9fU1lT Q0FMTChuciwgc3ltKSBleHRlcm4gbG9uZyBfX3g2NF8jI3N5bShjb25zdCBzdHJ1Y3QgcHRf cmVncyAqKTsKICNkZWZpbmUgX19TWVNDQUxMX05PUkVUVVJOKG5yLCBzeW0pIGV4dGVybiBs b25nIF9fbm9yZXR1cm4gX194NjRfIyNzeW0oY29uc3Qgc3RydWN0IHB0X3JlZ3MgKik7CiAj aW5jbHVkZSA8YXNtL3N5c2NhbGxzXzY0Lmg+CkBAIC0zMiw3ICs0MCw3IEBAIGNvbnN0IHN5 c19jYWxsX3B0cl90IHN5c19jYWxsX3RhYmxlW10gPSB7CiAjdW5kZWYgIF9fU1lTQ0FMTAog CiAjZGVmaW5lIF9fU1lTQ0FMTChuciwgc3ltKSBjYXNlIG5yOiByZXR1cm4gX194NjRfIyNz eW0ocmVncyk7Ci1sb25nIHg2NF9zeXNfY2FsbChjb25zdCBzdHJ1Y3QgcHRfcmVncyAqcmVn cywgdW5zaWduZWQgaW50IG5yKQorbG9uZyBhbGlnbl9mdW5jKDMyKSB4NjRfc3lzX2NhbGwo Y29uc3Qgc3RydWN0IHB0X3JlZ3MgKnJlZ3MsIHVuc2lnbmVkIGludCBucikKIHsKIAlzd2l0 Y2ggKG5yKSB7CiAJI2luY2x1ZGUgPGFzbS9zeXNjYWxsc182NC5oPgpAQCAtNDEsNyArNDks NyBAQCBsb25nIHg2NF9zeXNfY2FsbChjb25zdCBzdHJ1Y3QgcHRfcmVncyAqcmVncywgdW5z aWduZWQgaW50IG5yKQogfQogCiAjaWZkZWYgQ09ORklHX1g4Nl9YMzJfQUJJCi1sb25nIHgz Ml9zeXNfY2FsbChjb25zdCBzdHJ1Y3QgcHRfcmVncyAqcmVncywgdW5zaWduZWQgaW50IG5y KQorbG9uZyBhbGlnbl9mdW5jKDMyKSB4MzJfc3lzX2NhbGwoY29uc3Qgc3RydWN0IHB0X3Jl Z3MgKnJlZ3MsIHVuc2lnbmVkIGludCBucikKIHsKIAlzd2l0Y2ggKG5yKSB7CiAJI2luY2x1 ZGUgPGFzbS9zeXNjYWxsc194MzIuaD4K --------------QW1Vv0qTW0Zy5HelAHfOJAAK--