From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD95DC282CC for ; Tue, 5 Feb 2019 22:54:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 84CE1217F9 for ; Tue, 5 Feb 2019 22:54:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730266AbfBEWyM (ORCPT ); Tue, 5 Feb 2019 17:54:12 -0500 Received: from mga12.intel.com ([192.55.52.136]:13884 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726742AbfBEWyM (ORCPT ); Tue, 5 Feb 2019 17:54:12 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Feb 2019 14:54:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,337,1544515200"; d="scan'208";a="140983146" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga002.fm.intel.com with ESMTP; 05 Feb 2019 14:54:11 -0800 From: Rick Edgecombe To: daniel@iogearbox.net, ast@fb.com Cc: netdev@vger.kernel.org, ard.biesheuvel@linaro.org, dave.hansen@intel.com, kristen@linux.intel.com, Rick Edgecombe Subject: [RFC PATCH 0/4] Initial support for allocating BPF JITs in vmalloc for x86 Date: Tue, 5 Feb 2019 14:50:59 -0800 Message-Id: <20190205225103.28296-1-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This introduces a new capability for BPF program JIT's to be located in vmalloc space on x86_64. This can serve as a backup area for CONFIG_BPF_JIT_ALWAYS_ON in case an unprivileged app uses all of the module space allowed by bpf_jit_limit. In order to allow for calls from the increased distance of vmalloc from kernel/module space, relative calls are emitted as full indirect calls if the maximum relative call distance is exceeded. So the resulting performance of call BPF instructions in this case is similar to the BPF interpreter. Below is the results for the BPF unit test "BPF_MAXINSNS: Call heavy transformations": Config Duration 1 Duration 2 -------------------------------------------------- JIT Modules 13304 14224 JIT Vmalloc 33711 26877 Interpreter 31931 28180 Retpoline JIT Modules 14517 13375 Retpoline JIT Vmalloc 65739 60281 Retpoline Interpreter 63097 61141 Since this benchmark is made up of only calls to the kernel text, it suggests real world performance shouldn't ever be worse than the interpreter. The logic for when to use vmalloc, is that we use module space unless it is full, or, we are not CAP_SYS_ADMIN and bpf_jit_limit is exceeded. So vmalloc is only used when either the insertion would fail, or BPF would fallback to the interpreter. So there should be no run time regression when used in these scenarios. Todo: - This passes all the unit tests, but could use further verification that the compiler will still always converge. - Fix ARM after "bpf: Charge bpf jit limit in bpf_jit_alloc_exec" change - Update documentation for bpf_jit_limit Possible future optimizations: - Look at re-mapping some text in vmalloc so that calls can be in relative jump range. For example, a BPF library program could maybe be re-mapped multiple times so that a copy is always near the caller and so we could use the faster calls. Rick Edgecombe (4): bpf, x64: Implement BPF call retpoline bpf, x64: Increase distance for bpf calls bpf: Charge bpf jit limit in bpf_jit_alloc_exec bpf, x64: Enable unprivlidged jit in vmalloc arch/x86/include/asm/nospec-branch.h | 45 +++++++- arch/x86/net/bpf_jit_comp.c | 149 ++++++++++++++++++++++----- include/linux/filter.h | 3 + kernel/bpf/core.c | 20 ++-- 4 files changed, 183 insertions(+), 34 deletions(-) -- 2.17.1