From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37384C5478C for ; Wed, 28 Feb 2024 23:27:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:To :From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=OiHCg6rVQ2xbOspiLK7YgWipAuvt4IxH+b1Qy3DoESE=; b=wEAvz/fCdcWv18 iXyhWB0eMMSxbVLXg1/Jcf9ZgzfaMmkvYx1iNfhjB5s+RCktUtqkM372vObGoxzJcl5LFPJzcrc7K NrCd7PNFyZybfwlD2Qmpr6qMg7Z25P5Q2UmRSwh1WpPXlcBwWl9QLTrHYnZupp5X97XmK6qZg/nbX 7CGjo9F38PqWfdC6p91QIjC3Ut4k2ocSrd/MZNpadaPyfl7uIjAPO3saLPcAr9IK7vN9Y1fM8xtuE vGME6xDdFUXwj0K0eQfris4exc0ZrewgaoE4ZCIoA8agbWYB9Juw0x4aQSZmBs61V8O2hV88uMti5 oSwmbMSz7tVsJ/gPs84w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfTKh-0000000BHZU-3k2p; Wed, 28 Feb 2024 23:27:27 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rfTKf-0000000BHYy-2Z5t for linux-arm-kernel@lists.infradead.org; Wed, 28 Feb 2024 23:27:26 +0000 Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-512f892500cso174101e87.3 for ; Wed, 28 Feb 2024 15:27:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709162843; x=1709767643; darn=lists.infradead.org; h=mime-version:message-id:date:subject:to:from:from:to:cc:subject :date:message-id:reply-to; bh=qUsI28IGlUJBAILlE5CNXqMn2Wot3QZGGWsRmezOGs8=; b=QQg2HZ9JRKuKbly5E40ycB44UpYGwCwqq2t1Q6cKLTA2NAtcnUOJxVLv06QlG2hry1 olY+u9CCRhkIse3fDpjB4SqBD8tkQvgS2joVnZRouyq1O4D3Y7CFfuucQyU7OA52PAEo Qs0Y5rCsUJ3ovUtBfjamlZCtiMxaPYps5hsz2pn0vas7LSfJH+1H4Ju0qIQL+nNiZiHV kvwXJ8Xj6DWhZijXyKdKaFmlkEP1I5KAyzaHH4BMOQpNh+8pKnbK2QlAOKmk5idrcLY2 0+WTnXOZq63InHZEZ5gnMOG7fW0hlYWfCw95FSD+5cPKhmGnQnWm7qM/5DGSLN6YVL+e JQ3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709162843; x=1709767643; h=mime-version:message-id:date:subject:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=qUsI28IGlUJBAILlE5CNXqMn2Wot3QZGGWsRmezOGs8=; b=iiNKuqyemYXOUVsLRGdxH0KgTdHMPUnpwon/RQbVgDCGirFASbSIav68B3eUaoNdjt yox+3GmocOyEiUo4LanS2hpKyGXoHAO1f4mD7PHlAgLPcX2u+WlfOWbkv+xjvmzJXGZL Uu9vfwPMahujvZYugVMomJHzJmU+F2EAqA7ayyFdzoupNdBiBl9j22xGqTSOfl06wgNK bJthqnD82qgQ1qqt5PBn9ClEOt9YBSsmWE6x7rW6k3iUUaV5Q8MZYA1r0jZyIRdyoFxx hvwnrLvc0xqhC0BAvPcbSKVeA2p7F2ayeLnD/orTmdg6nAD4hp4dmCWptANspT6m6UQQ hphA== X-Forwarded-Encrypted: i=1; AJvYcCWsKGKb8WFgOfVtr9ebo/a4DiXi4SgmcT2N+TvYP3DMpWsVGN7b8cy9tRsCE/aMsbQfzsNcy3U3x8ZuWjyNEeUDZQt5fcky91QyEJulp/WSkHt9ug8= X-Gm-Message-State: AOJu0YycvrKTjIPvwMmOJr3bgyBGYAdApRKw/FP5IvL5ovuA6KqphOSu JcNt1C+1fpkEFuUpoJcJWAReWOZq1keRHyGCHxC+9fS9YY1QSrQ5ryr85/ZB9aEe6g== X-Google-Smtp-Source: AGHT+IEUDNBkFdHTxEGr88/rougzxjN7wW1dNLNTE4EokEqvE3FvIDWV4qKKcWv4RZPmxcXgd163Ng== X-Received: by 2002:a05:6512:605:b0:513:1a02:7304 with SMTP id b5-20020a056512060500b005131a027304mr202101lfe.54.1709162842814; Wed, 28 Feb 2024 15:27:22 -0800 (PST) Received: from localhost (54-240-197-231.amazon.com. [54.240.197.231]) by smtp.gmail.com with ESMTPSA id m19-20020a056000181300b0033d3b8820f8sm51909wrh.109.2024.02.28.15.27.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Feb 2024 15:27:22 -0800 (PST) From: puranjay12@gmail.com To: catalin.marinas@arm.com, ast@kernel.org, daniel@iogearbox.net, mark.rutland@arm.com, broonie@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: arm64: Supporting DYNAMIC_FTRACE_WITH_CALL_OPS with CLANG_CFI Date: Wed, 28 Feb 2024 23:27:03 +0000 Message-ID: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240228_152725_674306_C7FC6746 X-CRM114-Status: GOOD ( 16.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org I recently worked on allowing BPF programs to work properly on CLANG_CFI enabled kernels [1]. While doing this I found that fentry programs are failing to attach because DYNAMIC_FTRACE_WITH_CALL_OPS doesn't work with CLANG_CFI. Mark told me that the problem is that clang CFI places the type hash immediately before any pre-function NOPs, and so where some functions have pre-function NOPs and others do not, the type hashes are not at a consistent offset (and effectively the functions have different ABIs and cannot call one another) I tried enabling both Clang CFI and -fpatchable-function-entry=4,2 to see the behaviour and where this could fail. Here is an example: This is the disassembly of jump_label_cmp() that has two pre-function nops and the CFI hash before them. So, the hash is at (addr - 12). ffff80008033e9b0: 16c516ce [kCFI hash for 'static int jump_label_cmp(const void *a, const void *b)'] ffff80008033e9b4: d503201f nop ffff80008033e9b8: d503201f nop ffff80008033e9bc : ffff80008033e9bc: d503245f bti c ffff80008033e9c0: d503201f nop ffff80008033e9c4: d503201f nop [.....] The following is the disassembly of the sort_r() function that makes an indirect call to jump_label_cmp() but loads the CFI hash from (addr - 4) rather than (addr - 12). So, it is loading the nop instruction and not the hash. ffff80008084e19c : [.....] 0xffff80008084e454 <+696>: ldur w16, [x8, #-4] (#-4 here should be #-12) 0xffff80008084e458 <+700>: movk w17, #0x16ce 0xffff80008084e45c <+704>: movk w17, #0x16c5, lsl #16 0xffff80008084e460 <+708>: cmp w16, w17 0xffff80008084e464 <+712>: b.eq 0xffff80008084e46c // b.none 0xffff80008084e468 <+716>: brk #0x8228 0xffff80008084e46c <+720>: blr x8 This would cause a cfi exception. As I haven't spent more time trying to understand this, I am not aware how the compiler emits 2 nops before some functions and none for others. I would propose the following changes to the compiler that could fix this issue: 1. The kCFI hash should always be generated at func_addr - 4, this would make the calling code consistent. 2. The two(n) nops should be generated before the kCFI hash. We would modify the ftrace code to look for these nops at (fun_addr - 12) and (func_addr - 8) when CFI is enabled, and (func_addr - 8), (func_addr - 4) when CFI is disabled. The generated code could then look like: ffff80008033e9b0: d503201f nop ffff80008033e9b4: d503201f nop ffff80008033e9b8: 16c516ce kCFI hash ffff80008033e9bc : ffff80008033e9bc: d503245f bti c ffff80008033e9c0: d503201f nop ffff80008033e9c4: d503201f nop [.....] Note: I am overlooking the alignment requirements here, we might need to add another nop above the hash to make sure the top two nops are aligned at 8 bytes. I am not sure how useful this solution is, looking forward to hear from others who know more about this topic. Thanks, Puranjay [1] https://lore.kernel.org/bpf/20240227151115.4623-1-puranjay12@gmail.com/ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel