From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89F59C7619A for ; Sat, 15 Apr 2023 12:09:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCE0C6B0072; Sat, 15 Apr 2023 08:09:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C7E7B6B0075; Sat, 15 Apr 2023 08:09:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B46D26B0078; Sat, 15 Apr 2023 08:09:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A6B7F6B0072 for ; Sat, 15 Apr 2023 08:09:21 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 78E3340133 for ; Sat, 15 Apr 2023 12:09:21 +0000 (UTC) X-FDA: 80683505322.27.809D9D1 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by imf09.hostedemail.com (Postfix) with ESMTP id B67A214000E for ; Sat, 15 Apr 2023 12:09:19 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="M0DAP/83"; spf=pass (imf09.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681560559; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=OFJP1DEOM2hcDBxvWV93h2rtiTrUT+yDD0c0jto4Tbk=; b=1viiip8ICmiZBdiDnnJ2ZrYkEG+sJfTL83ZL5TJBjTjjQCEnBc+FWdTEcetVQVjRw+goem eLwN010OAcnwYIwTvILeYAciq/SkkuNzOaqdyJDFS0g0iqK2mkQc1UQCY9Ldz8huZLgZQY OYIEukY6CZBeYlabpV1JLPuuXwZI3lw= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="M0DAP/83"; spf=pass (imf09.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681560559; a=rsa-sha256; cv=none; b=VMh1x+y6FGwsxr70+Ya1TR8pC6lGiTDrncPnJUK98zOmxURvkqijYYNfBjGBnI+CG0lBpI q2nlVXf7M8IreCsLXiJvLbBQzC0Ief7n7rKuVOUAS+dz8e5NFpUn+ApcW1gRKk6GTMnfxn XSgGoh9Ed0HzveVe1K8Qvlz5dx+rQhE= Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-2ea1257d967so322430f8f.2 for ; Sat, 15 Apr 2023 05:09:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681560558; x=1684152558; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=OFJP1DEOM2hcDBxvWV93h2rtiTrUT+yDD0c0jto4Tbk=; b=M0DAP/83XHRXNFaFuUrIe0FPT2ywMdU7/EBmKQkTuzotA3+g/mX5EiP3BZ+gmRdcOl UFrBOLbbcBc2zRVa5VcBRxWVoQLbT3u4UDKUxj1j8r+KUEAcNw+jp5GMYy5Qna5WIi4n wCciW3dklDpS4KBECt4G0sWLch9GdrifFX0Y3oipbTaFgwH6rAobYlBjp6ZeYCChwHwu a93hq7U9JKqyU8tbsVUNx5Ic/yKw4fXOYjce49G0+fOVQ4A9axdGZ6eoY0Py0irGUfkF cTfQ/2fUz/BDqh2Z7ElN+Epnoix8a7JN3AI6y1EAcJNqOo2xQkm12U0oZBCONQAHFbKK eU/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681560558; x=1684152558; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OFJP1DEOM2hcDBxvWV93h2rtiTrUT+yDD0c0jto4Tbk=; b=RLaYinV6VfjjX/gLijKLoxfYlBgDTU97w18t+g4WmUBoNp9C84mb1i3EgN/8liy97g HzNWWFcFrRw90eYghLKwiL22naw8c4AgUd0MpNS8nOehZb9JskOlacYoe22gdOyNtODA WEs2QtZ/kcRE7amXlXNtTxl6QZPY3sJqPJPVkKxVHVe2X6qI1lWeFFoYtff/quviDof4 4M5Av1pFEuMxp9dol+0CFHe7oqgdzbpBA+XJ07QGGSTHTlxnZMilJb/6TTu3YyE73K6U 2QNZFpGApybr0CaeC53m1eCAud1hr8GCPXDoN1Z6pOH1yMNE+wtDtm5EreY7yQEPgEE9 ivqg== X-Gm-Message-State: AAQBX9d4fcM9afNcMrfhV7Db578n7BMZaW3slQCvRZ0U1AllsmSY6ZPw Ds5UmmrCseWEda2tKhT4xgvfkBLHkfQHEQ== X-Google-Smtp-Source: AKy350bdgIVIT8Ikiu4Ulq2mLTKwIF3hcqxe6rsYgcxgaWqBWXYlZO3ICi2NP+5izhbKbr9kn0X6yA== X-Received: by 2002:adf:e802:0:b0:2d1:7ade:aac with SMTP id o2-20020adfe802000000b002d17ade0aacmr1306465wrm.0.1681560557807; Sat, 15 Apr 2023 05:09:17 -0700 (PDT) Received: from lucifer.home ([2a00:23c5:dc8c:8701:1663:9a35:5a7b:1d76]) by smtp.googlemail.com with ESMTPSA id h12-20020adffa8c000000b002d322b9a7f5sm5657253wrr.88.2023.04.15.05.09.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 15 Apr 2023 05:09:17 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton Cc: Matthew Wilcox , David Hildenbrand , Lorenzo Stoakes Subject: [PATCH v3 0/7] remove the vmas parameter from GUP APIs Date: Sat, 15 Apr 2023 13:09:14 +0100 Message-Id: X-Mailer: git-send-email 2.40.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B67A214000E X-Rspam-User: X-Stat-Signature: 94bbyryt56ii9ych66pp9pn3mbap7k1j X-HE-Tag: 1681560559-228732 X-HE-Meta: U2FsdGVkX1/y3RoP1INrGOx/L0766KrtkVtAze2Ii1leNMppKWcYzLmnDrij4tKDVnFWLuWiTHk7wP3yArBG9nN7aP4CNtl8D89zYTcBzJcNgZaG4y2MIVtGUb/cpZ6evtbL0MBDYQIxkRZw3YYPlSmgF/Fg/eI085wLy6g7UMSJKPHPyODyR5BATpl+CZYcOUehkcPxCrGltNyAlJlt48QKdDFCQ4DgXpI8QK7g0oBRxjz8vt0hl/atdB5rhyONjWxjZ7uwlemdgazsyVHCkuC3ByrbkjAIRa2/gjdCt3KwkY6O/vehlga9ALLTD7cJ3WNHxp0TP4M4UK9XaENCr7WODmD/pTvWojxRC4ZyvwP4tuSziqgA+R7SkMNo6pJW8ics64ZNJeNrl5uTIm7OrGLnah8KlUUfpzdiXov303xyebTmuwlZCAOaNxqqgV79W4v0j3cvClqxf+5Y8VxeQaOpmBCxCy+osx8iTsHfP29cQJ9n+c6wcbfy54Oco3EgW58Ni/jHRA5iUcUYAlcg/uMKwXaKHCUqm31RWcAv2lJvZsOiSuSdxQ8CuWTWZHvmXNkOW84UTncvFEiZWg84i3itZTIjwEmGsdBo5fgf/lVoEf3ibEccQWGUEZIEyKi+7GOhvKiIOQKcw4malB8RYXR+b13752akeTS0QgCU1EvyG3I95JBEYzCbnhn3fZBmm4bm9fIQ9pYXz5Xh+rBKJrs74Il18IBLQQnDfvz7DFawomkalywe1ERgDIFiNPxD3wApZWfhz/eZD6Emk42DmqopP7FXtd7JLt6QUUMBrdOiWXEnSR6BhTvGS+Jd+9sDMq6pHrD5ZfOsf+Nwy5BBzJWdiRfQ7U4XCNys6p/BCa/UVykJ3S83IYDzA2/wCIDR05Gdw9mv6Q/ZKkz/t1ZNsRE2A6Ss/w+13FVhw2k/YIAo4/1FMICdNreWSjJrihbkRBedsYRhK/6SVJZzUUp 8pGe1kcI Lkpz79SUjs/nMNdcV7Hod8WvFquTBHF4XuNli/Gx4nO9U8udMPZBjPY7CjSjRu41BUXtwMhmd+PWGYxNQ17gqcjTZMmMAgtMSWMHrueHlO9DOxaXX9LDFcsHBjkaCjVJ6bbLNLs0wMnRjV4ymsA5raX1DEN+fMgCrDRHCyfx2HXl6JROoYiZ6huSEH4w6Xdcylq2XMZ9Vt1NQHThoClTLgXWFjohwd+x6sGjjfEKPNWskDqJ1aiQrJ9/JVnmt8TAH/i0NiifLqL4CztEJNlEuS4qZxtALZTZ6C5oXUYrIU14lMAJWGjrXN2NOCB2c72psPEvvM1jTJu7fGGo8iQwjUbgoslE7eI0U/ydeOx7TATfqHiBWzIX6ByQugQpSvOd7oUslDgp9dssgskmRhWa7FmBjZMoTXBpi0JiPLlenN3OeVHVffkO9+Zmis/IaOLD2Ear/KavPDxk12eBMVXtfHK4yhA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: (pin_/get)_user_pages[_remote]() each provide an optional output parameter for an array of VMA objects associated with each page in the input range. These provide the means for VMAs to be returned, as long as mm->mmap_lock is never released during the GUP operation (i.e. the internal flag FOLL_UNLOCKABLE is not specified). In addition, these VMAs have also to only be accessed under the mmap_lock, and become invalidated the moment it is released. The vast majority of invocations do not use this functionality and of those that do, all but one retrieve a single VMA to perform checks upon. It is not egregious in the single VMA cases to simply replace the operation with a vma_lookup(). In these cases we duplicate the (fast) lookup on a slow path already under the mmap_lock. The special case is io_uring, where io_pin_pages() specifically needs to assert that all the VMAs possess the same vm->vm_file (possibly NULL) and they are either anonymous or hugetlb pages. To continue to provide this functionality, we introduce the FOLL_SAME_PAGE flag which asserts that the vma->vm_file remains the same throughout, erroring out if this is not the case. We can then replace the io_uring case by passing FOLL_SAME_FILE and looking up the first VMA manually and performing the required checks on this alone. The combination of the two amount to the same checks being performed (and avoids an allocation). Eliminating this parameter eliminates an entire class of errors - the vmas array used to become a set of dangling pointers if access after release of mmap_lock was attempted, this is simply no longer possible. In addition the API is simplified and now clearly expresses what it is for - applying the specified GUP flags and (if pinning) returning pinned pages. This change additionally opens the door to further potential improvements in GUP and the possible marrying of disparate code paths. I have run the gup_test and a simple io_uring program which exercises the use of FOLL_SAME_PAGE with no issues. Thanks to Matthew Wilcox for suggesting this refactoring! v3: - Always explicitly handle !vma cases, feeding back an error to the user if appropriate, indicating the operation did not completely succeed if not and always with a warning since these conditions should be impossible. v2: - Only lookup the VMA if the pin succeeded (other than __access_remote_vm() which has different semantics) - Be pedantically careful about ensuring that under no circumstances can we fail to unpin a page https://lore.kernel.org/linux-mm/cover.1681547405.git.lstoakes@gmail.com/ v1: https://lore.kernel.org/linux-mm/cover.1681508038.git.lstoakes@gmail.com/ Lorenzo Stoakes (7): mm/gup: remove unused vmas parameter from get_user_pages() mm/gup: remove unused vmas parameter from pin_user_pages_remote() mm/gup: remove vmas parameter from get_user_pages_remote() mm/gup: introduce the FOLL_SAME_FILE GUP flag io_uring: rsrc: use FOLL_SAME_FILE on pin_user_pages() mm/gup: remove vmas parameter from pin_user_pages() mm/gup: remove vmas array from internal GUP functions arch/arm64/kernel/mte.c | 7 +- arch/powerpc/mm/book3s64/iommu_api.c | 2 +- arch/s390/kvm/interrupt.c | 2 +- arch/x86/kernel/cpu/sgx/ioctl.c | 2 +- drivers/gpu/drm/radeon/radeon_ttm.c | 2 +- drivers/infiniband/hw/qib/qib_user_pages.c | 2 +- drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +- drivers/infiniband/sw/siw/siw_mem.c | 2 +- drivers/iommu/iommufd/pages.c | 4 +- drivers/media/v4l2-core/videobuf-dma-sg.c | 2 +- drivers/misc/sgi-gru/grufault.c | 2 +- drivers/vdpa/vdpa_user/vduse_dev.c | 2 +- drivers/vfio/vfio_iommu_type1.c | 2 +- drivers/vhost/vdpa.c | 2 +- fs/exec.c | 2 +- include/linux/hugetlb.h | 10 +- include/linux/mm.h | 10 +- include/linux/mm_types.h | 2 + io_uring/rsrc.c | 40 +++---- kernel/events/uprobes.c | 14 ++- mm/gup.c | 121 ++++++++------------- mm/gup_test.c | 14 +-- mm/hugetlb.c | 24 ++-- mm/memory.c | 8 +- mm/process_vm_access.c | 2 +- mm/rmap.c | 2 +- net/xdp/xdp_umem.c | 2 +- security/tomoyo/domain.c | 2 +- virt/kvm/async_pf.c | 3 +- virt/kvm/kvm_main.c | 2 +- 30 files changed, 132 insertions(+), 161 deletions(-) -- 2.40.0