From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AF551CAA87 for ; Tue, 11 Mar 2025 15:49:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741708197; cv=none; b=lpSsiKO/944RGgnpNbQVUfFcaDkRxuxtpbcp+PpiJKh03LswSYCif1+0IYL3x6bJktMaBfxGrCLfmcMEDyb2lpa+Y3Mw8g+ma/tlt6pMmJCKvFi4mZ0yx5w/xhB1koWnPhtaKQnFnyi0cfsZK0nOK6AvV6IR6w3f76i9ih9p0Ok= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741708197; c=relaxed/simple; bh=PzFqfoRtY4n2SOUoLNd8OHnIMa/d6SS3tquxpWs0kpA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Ld5c56lI85HKaptwZConDHagN566ED2D+XHwZD1byqrxoktHQGJFVVYBks0W/58V/tCnySqQJjqwXjedq75Mdcu+xf0rNBULjoeVJdCFvNdzIlFUXGu5ujZfpqPV7j5cVCW42oq5Zzf/GvM4cZ074nLMVxk3BxooUN48x+uwMlE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=kiEgAGRE; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="kiEgAGRE" Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 52B7xUNZ006196; Tue, 11 Mar 2025 08:49:13 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=tAiN/cUXd5vSssqq46CTEACFwTd9FaSY1CA4OHp/kAo=; b=kiEgAGREhZwN RoAnhzFkNWC3lHVfMCJffgpeTWUhdiAvWUs7qOGPzxD5Hgg+/7h4HJjkeUYB7/Qo +pviD+p7CCqTolJhaTrAr5ReDnIx9OMY8LcYO932c61xFynfeCx3Uuwe7qdVoXok kp6im0Q5RhAHDCdUs0R0M4W/0eFaNeXow5qqGB6ohG7oHOTLy0i4Ye6YR2FlDmF9 4kjYaui+xTfrcguz8wHlBvjLmNE5fcpMgSC5LNURirlX969WnwXHTh/NqbFM7yVY 8oMbWQLfW4eUfFJ2/gyJJ8nC9ng43JKqzGy/O9iv/ckFTjnzoxWjcdxgeck0gtTJ hU1RPLN7ww== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 45ah8rttkv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 11 Mar 2025 08:49:13 -0700 (PDT) Received: from devvm4158.cln0.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c08b:78::2ac9) with Microsoft SMTP Server id 15.2.1544.14; Tue, 11 Mar 2025 15:49:09 +0000 From: Vadim Fedorenko To: Borislav Petkov , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Eduard Zingerman , Thomas Gleixner , Yonghong Song , Vadim Fedorenko , Mykola Lysenko CC: , , Peter Zijlstra , Vadim Fedorenko , Martin KaFai Lau Subject: [PATCH bpf-next v10 3/4] selftests/bpf: add selftest to check bpf_get_cpu_time_counter jit Date: Tue, 11 Mar 2025 08:48:49 -0700 Message-ID: <20250311154850.3616840-4-vadfed@meta.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250311154850.3616840-1-vadfed@meta.com> References: <20250311154850.3616840-1-vadfed@meta.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: uabo5GmSAtSK1yOW5GS6Zb67LRHgDu3u X-Proofpoint-ORIG-GUID: uabo5GmSAtSK1yOW5GS6Zb67LRHgDu3u X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1093,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-03-11_04,2025-03-11_02,2024-11-22_01 bpf_get_cpu_time_counter() is replaced with rdtsc instruction on x86_64. Add tests to check that JIT works as expected. Acked-by: Eduard Zingerman Signed-off-by: Vadim Fedorenko --- .../selftests/bpf/prog_tests/verifier.c | 2 + .../selftests/bpf/progs/verifier_cpu_cycles.c | 104 ++++++++++++++++++ 2 files changed, 106 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c index e66a57970d28..d5e7e302a344 100644 --- a/tools/testing/selftests/bpf/prog_tests/verifier.c +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c @@ -102,6 +102,7 @@ #include "verifier_xdp_direct_packet_access.skel.h" #include "verifier_bits_iter.skel.h" #include "verifier_lsm.skel.h" +#include "verifier_cpu_cycles.skel.h" #include "irq.skel.h" #define MAX_ENTRIES 11 @@ -236,6 +237,7 @@ void test_verifier_bits_iter(void) { RUN(verifier_bits_iter); } void test_verifier_lsm(void) { RUN(verifier_lsm); } void test_irq(void) { RUN(irq); } void test_verifier_mtu(void) { RUN(verifier_mtu); } +void test_verifier_cpu_cycles(void) { RUN(verifier_cpu_cycles); } static int init_test_val_map(struct bpf_object *obj, char *map_name) { diff --git a/tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c b/tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c new file mode 100644 index 000000000000..5b62e3690362 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2022 Meta Inc. */ +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" + +extern u64 bpf_cpu_time_counter_to_ns(u64 cycles) __weak __ksym; +extern u64 bpf_get_cpu_time_counter(void) __weak __ksym; + +SEC("syscall") +__arch_x86_64 +__xlated("0: call kernel-function") +__naked int bpf_rdtsc(void) +{ + asm volatile( + "call %[bpf_get_cpu_time_counter];" + "exit" + : + : __imm(bpf_get_cpu_time_counter) + : __clobber_all + ); +} + +SEC("syscall") +__arch_x86_64 +/* program entry for bpf_rdtsc_jit_x86_64(), regular function prologue */ +__jited(" endbr64") +__jited(" nopl (%rax,%rax)") +__jited(" nopl (%rax)") +__jited(" pushq %rbp") +__jited(" movq %rsp, %rbp") +__jited(" endbr64") +/* save RDX in R11 as it will be overwritten */ +__jited(" movq %rdx, %r11") +/* lfence may not be executed depending on cpu features */ +__jited(" {{(lfence|)}}") +__jited(" rdtsc") +/* combine EDX:EAX into RAX */ +__jited(" shlq ${{(32|0x20)}}, %rdx") +__jited(" orq %rdx, %rax") +/* restore RDX from R11 */ +__jited(" movq %r11, %rdx") +__jited(" leave") +__naked int bpf_rdtsc_jit_x86_64(void) +{ + asm volatile( + "call %[bpf_get_cpu_time_counter];" + "exit" + : + : __imm(bpf_get_cpu_time_counter) + : __clobber_all + ); +} + +SEC("syscall") +__arch_x86_64 +__xlated("0: r1 = 42") +__xlated("1: call kernel-function") +__naked int bpf_cyc2ns(void) +{ + asm volatile( + "r1=0x2a;" + "call %[bpf_cpu_time_counter_to_ns];" + "exit" + : + : __imm(bpf_cpu_time_counter_to_ns) + : __clobber_all + ); +} + +SEC("syscall") +__arch_x86_64 +/* program entry for bpf_rdtsc_jit_x86_64(), regular function prologue */ +__jited(" endbr64") +__jited(" nopl (%rax,%rax)") +__jited(" nopl (%rax)") +__jited(" pushq %rbp") +__jited(" movq %rsp, %rbp") +__jited(" endbr64") +/* save RDX in R11 as it will be overwritten */ +__jited(" movabsq $0x2a2a2a2a2a, %rdi") +__jited(" imulq ${{.*}}, %rdi, %rax") +__jited(" shrq ${{.*}}, %rax") +__jited(" leave") +__naked int bpf_cyc2ns_jit_x86(void) +{ + asm volatile( + "r1=0x2a2a2a2a2a ll;" + "call %[bpf_cpu_time_counter_to_ns];" + "exit" + : + : __imm(bpf_cpu_time_counter_to_ns) + : __clobber_all + ); +} + +void rdtsc(void) +{ + bpf_get_cpu_time_counter(); + bpf_cpu_time_counter_to_ns(42); +} + +char _license[] SEC("license") = "GPL"; -- 2.47.1