From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9444BC11F68 for ; Fri, 2 Jul 2021 07:01:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0A7CA61413 for ; Fri, 2 Jul 2021 07:01:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A7CA61413 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7818E6B0011; Fri, 2 Jul 2021 03:01:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 759B56B0036; Fri, 2 Jul 2021 03:01:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5846F6B005D; Fri, 2 Jul 2021 03:01:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 3857C6B0011 for ; Fri, 2 Jul 2021 03:01:28 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D5C0982E923E for ; Fri, 2 Jul 2021 07:01:27 +0000 (UTC) X-FDA: 78316751814.28.475AB16 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by imf22.hostedemail.com (Postfix) with ESMTP id 830DF1988 for ; Fri, 2 Jul 2021 07:01:27 +0000 (UTC) Received: by mail-pj1-f48.google.com with SMTP id 22-20020a17090a0c16b0290164a5354ad0so8547066pjs.2 for ; Fri, 02 Jul 2021 00:01:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=pOxwy4S8+CMaHsFPZCERpcLAZFzgkbgCU7ysS2kserA=; b=C7iqWm/wesRNy/TjG2rGEA+W4kQ+TyHVuuL7z1F62PhJmxaaEoArlcQc84EmSneSqz iznC5OCPYsOsUJxJhgRSMTidOscJf/aghiOmDfzN8vcOyS9Ktom0AAY3oZkSKKjSXO1d bNX6lm2SULxR5MVs+e4vybMcekoS7NI2FBZQjmcpcDVbq1yNnSJqb1zN5kPr2Z1pUbNk A5G6ojm9VhQ8EdJjli5tjWX9R26cjvQG9ITwcfWD6VHn+xMCAGOok6yLzhJ4HNTQ0yGZ /Y/LCtDHcowpYrJJLDpAoIzycnxxFSBNvQyxIEhjKJ4y6wPKqpXxQSs7x9PXOvzmkcds oZlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=pOxwy4S8+CMaHsFPZCERpcLAZFzgkbgCU7ysS2kserA=; b=rfaJnOcVhP/nMryebS8NOzhoeIG9ui7YYN/1HF8DAIG7htlGFMcJ4PaY+VlYwBUIGc /6I/LKVQwUo6D7N0C/tDReRyoEA9H9lo+AWmH7iSnVHScyPOazvX8c/a288DAc4zTSnZ ikE3ArgxpxI6oKTGsQxuaiP3LFhH/gho461/VriiXnyERpZQJDIpchHlaEGMd61I6zYa Sj3/dmFavtuIOyKO5YngMJnEnAB87NQo1VHuNpua19yEJl3Af+T1+Rv2GhwMh6nq6zxW zsIlTsuB0eslsXxAHNn/JnI3LLtWlv2Wm9kK8EyZ2w1Xyw1wfsJrx8uI1/J4BQ8cqkc/ pRUw== X-Gm-Message-State: AOAM532P6ybvS+IYxp0XBNmFenurZmKI+DxXKBsH1pOyFvU2GcsXi4ty Z4JzPAVvKERMlMxHBPX0SZ8= X-Google-Smtp-Source: ABdhPJyfi/fuR+AXjHafBzne+j9YCvbhOaUZINfEYssRYGLtqsbz0Nrvakl5C11G5EnSvNYHMM3PVw== X-Received: by 2002:a17:90b:4c4b:: with SMTP id np11mr13875779pjb.125.1625209286236; Fri, 02 Jul 2021 00:01:26 -0700 (PDT) Received: from gmail.com ([2601:600:8500:5f14:d627:c51e:516e:a105]) by smtp.gmail.com with ESMTPSA id p14sm2378495pgb.2.2021.07.02.00.01.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jul 2021 00:01:25 -0700 (PDT) Date: Thu, 1 Jul 2021 23:57:53 -0700 From: Andrei Vagin To: Jann Horn Cc: kernel list , Linux API , linux-um@lists.infradead.org, criu@openvz.org, avagin@google.com, Andrew Morton , Andy Lutomirski , Anton Ivanov , Christian Brauner , Dmitry Safonov <0x7f454c46@gmail.com>, Ingo Molnar , Jeff Dike , Mike Rapoport , Michael Kerrisk , Oleg Nesterov , Peter Zijlstra , Richard Weinberger , Thomas Gleixner , linux-mm@kvack.org Subject: Re: [PATCH 0/4 POC] Allow executing code and syscalls in another address space Message-ID: References: <20210414055217.543246-1-avagin@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 830DF1988 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="C7iqWm/w"; spf=pass (imf22.hostedemail.com: domain of avagin@gmail.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=avagin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: h9z4czxczwombczs1bs1zun1x5du3u3o X-HE-Tag: 1625209287-612065 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Apr 14, 2021 at 08:46:40AM +0200, Jann Horn wrote: > On Wed, Apr 14, 2021 at 7:59 AM Andrei Vagin wrote: > > We already have process_vm_readv and process_vm_writev to read and wr= ite > > to a process memory faster than we can do this with ptrace. And now i= t > > is time for process_vm_exec that allows executing code in an address > > space of another process. We can do this with ptrace but it is much > > slower. > > > > =3D Use-cases =3D >=20 > It seems to me like your proposed API doesn't really fit either one of > those usecases well... >=20 > > Here are two known use-cases. The first one is =E2=80=9Capplication k= ernel=E2=80=9D > > sandboxes like User-mode Linux and gVisor. In this case, we have a > > process that runs the sandbox kernel and a set of stub processes that > > are used to manage guest address spaces. Guest code is executed in th= e > > context of stub processes but all system calls are intercepted and > > handled in the sandbox kernel. Right now, these sort of sandboxes use > > PTRACE_SYSEMU to trap system calls, but the process_vm_exec can > > significantly speed them up. >=20 > In this case, since you really only want an mm_struct to run code > under, it seems weird to create a whole task with its own PID and so > on. It seems to me like something similar to the /dev/kvm API would be > more appropriate here? Implementation options that I see for that > would be: >=20 > 1. mm_struct-based: > a set of syscalls to create a new mm_struct, > change memory mappings under that mm_struct, and switch to it I like the idea to have a handle for mm. Instead of pid, we will pass this handle to process_vm_exec. We have pidfd for processes and we can introduce mmfd for mm_struct. > 2. pagetable-mirroring-based: > like /dev/kvm, an API to create a new pagetable, mirror parts of > the mm_struct's pagetables over into it with modified permissions > (like KVM_SET_USER_MEMORY_REGION), > and run code under that context. > page fault handling would first handle the fault against mm->pgd > as normal, then mirror the PTE over into the secondary pagetables= . > invalidation could be handled with MMU notifiers. > I found this idea interesting and decided to look at it more closely. After reading the kernel code for a few days, I realized that it would not be easy to implement something like this, but more important is that I don=E2=80=99t understand what problem it solves. Will it simplify the user-space code? I don=E2=80=99t think so. Will it improve performance? I= t is unclear for me too. First, in the KVM case, we have a few big linear mappings and need to support one =E2=80=9Cshadow=E2=80=9D address space. In the case of sandbo= xes, we can have a tremendous amount of mappings and many address spaces that we need to manage. Memory mappings will be mapped with different addresses in a supervisor address space and =E2=80=9Cguest=E2=80=9D address spaces.= If guest address spaces will not have their mm_structs, we will need to reinvent vma-s in some form. If guest address spaces have mm_structs, this will look similar to https://lwn.net/Articles/830648/. Second, each pagetable is tied up with mm_stuct. You suggest creating new pagetables that will not have their mm_struct-s (sorry if I misunderstood something). I am not sure that it will be easy to implement. How many corner cases will be there? As for page faults in a secondary address space, we will need to find a fault address in the main address space, handle the fault there and then mirror the PTE to the secondary pagetable. Effectively, it means that page faults will be handled in two address spaces. Right now, we use memfd and shared mappings. It means that each fault is handled only in one address space, and we map a guest memory region to the supervisor address space only when we need to access it. A large portion of guest anonymous memory is never mapped to the supervisor address space. Will an overhead of mirrored address spaces be smaller than memfd shared mappings? I am not sure. Third, this approach will not get rid of having process_vm_exec. We will need to switch to a guest address space with a specified state and switch back on faults or syscalls. If the main concern is the ability to run syscalls on a remote mm, we can think about how to fix this. I see two ways what we can do here: * Specify the exact list of system calls that are allowed. The first three candidates are mmap, munmap, and vmsplice. * Instead of allowing us to run system calls, we can implement this in the form of commands. In the case of sandboxes, we need to implement only two commands to create and destroy memory mappings in a target address space. Thanks, Andrei