From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 512FEC4332B for ; Tue, 26 Jan 2021 11:59:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2680E23120 for ; Tue, 26 Jan 2021 11:59:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405185AbhAZL7i (ORCPT ); Tue, 26 Jan 2021 06:59:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:27261 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2405274AbhAZL6m (ORCPT ); Tue, 26 Jan 2021 06:58:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611662235; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qjJ7EBIraH0kcBiHWVTNphRxbXFU8GNfvJrsoVivTWk=; b=V8WD8NCMWlFDx9o6YzcO4I4dENQNX15Lz56GUb71wwvfnEUCghoGq2WMCpCus3UksH4ryw zGNU8VB40n7JCwxI/iYewHuZuQskrP2EPqwOLVMVQm7Din2QT7woVj7CSLOVIA2dubHziT gr4NL1JD2oKEm1+KynWmZgRhW0U44EU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-Ryg-fxAdMkms-vm90M751Q-1; Tue, 26 Jan 2021 06:57:11 -0500 X-MC-Unique: Ryg-fxAdMkms-vm90M751Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9F94E107ACF6; Tue, 26 Jan 2021 11:57:06 +0000 (UTC) Received: from [10.36.114.192] (ovpn-114-192.ams2.redhat.com [10.36.114.192]) by smtp.corp.redhat.com (Postfix) with ESMTP id B16B65D751; Tue, 26 Jan 2021 11:56:49 +0000 (UTC) To: Michal Hocko , Mike Rapoport Cc: Andrew Morton , Alexander Viro , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Christopher Lameter , Dan Williams , Dave Hansen , Elena Reshetova , "H. Peter Anvin" , Ingo Molnar , James Bottomley , "Kirill A. Shutemov" , Matthew Wilcox , Mark Rutland , Mike Rapoport , Michael Kerrisk , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Rick Edgecombe , Roman Gushchin , Shakeel Butt , Shuah Khan , Thomas Gleixner , Tycho Andersen , Will Deacon , linux-api@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-nvdimm@lists.01.org, linux-riscv@lists.infradead.org, x86@kernel.org, Hagen Paul Pfeifer , Palmer Dabbelt References: <20210121122723.3446-1-rppt@kernel.org> <20210121122723.3446-8-rppt@kernel.org> <20210126114657.GL827@dhcp22.suse.cz> From: David Hildenbrand Organization: Red Hat GmbH Subject: Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation Message-ID: <303f348d-e494-e386-d1f5-14505b5da254@redhat.com> Date: Tue, 26 Jan 2021 12:56:48 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <20210126114657.GL827@dhcp22.suse.cz> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-api@vger.kernel.org On 26.01.21 12:46, Michal Hocko wrote: > On Thu 21-01-21 14:27:19, Mike Rapoport wrote: >> From: Mike Rapoport >> >> Removing a PAGE_SIZE page from the direct map every time such page is >> allocated for a secret memory mapping will cause severe fragmentation of >> the direct map. This fragmentation can be reduced by using PMD-size pages >> as a pool for small pages for secret memory mappings. >> >> Add a gen_pool per secretmem inode and lazily populate this pool with >> PMD-size pages. >> >> As pages allocated by secretmem become unmovable, use CMA to back large >> page caches so that page allocator won't be surprised by failing attempt to >> migrate these pages. >> >> The CMA area used by secretmem is controlled by the "secretmem=" kernel >> parameter. This allows explicit control over the memory available for >> secretmem and provides upper hard limit for secretmem consumption. > > OK, so I have finally had a look at this closer and this is really not > acceptable. I have already mentioned that in a response to other patch > but any task is able to deprive access to secret memory to other tasks > and cause OOM killer which wouldn't really recover ever and potentially > panic the system. Now you could be less drastic and only make SIGBUS on > fault but that would be still quite terrible. There is a very good > reason why hugetlb implements is non-trivial reservation system to avoid > exactly these problems. > > So unless I am really misreading the code > Nacked-by: Michal Hocko > > That doesn't mean I reject the whole idea. There are some details to > sort out as mentioned elsewhere but you cannot really depend on > pre-allocated pool which can fail at a fault time like that. So, to do it similar to hugetlbfs (e.g., with CMA), there would have to be a mechanism to actually try pre-reserving (e.g., from the CMA area), at which point in time the pages would get moved to the secretmem pool, and a mechanism for mmap() etc. to "reserve" from these secretmem pool, such that there are guarantees at fault time? What we have right now feels like some kind of overcommit (reading, as overcommiting huge pages, so we might get SIGBUS at fault time). TBH, the SIGBUS thingy doesn't sound terrible to me - if this behavior is to be expected right now by applications using it and they can handle it - no guarantees. I fully agree that some kind of reservation/guarantee mechanism would be preferable. -- Thanks, David / dhildenb From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DEC9C433E0 for ; Tue, 26 Jan 2021 11:57:19 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 54FDA2311D for ; Tue, 26 Jan 2021 11:57:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 54FDA2311D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 09277100EB829; Tue, 26 Jan 2021 03:57:19 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=63.128.21.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=david@redhat.com; receiver= Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id F3ECA100EB825 for ; Tue, 26 Jan 2021 03:57:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611662235; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qjJ7EBIraH0kcBiHWVTNphRxbXFU8GNfvJrsoVivTWk=; b=V8WD8NCMWlFDx9o6YzcO4I4dENQNX15Lz56GUb71wwvfnEUCghoGq2WMCpCus3UksH4ryw zGNU8VB40n7JCwxI/iYewHuZuQskrP2EPqwOLVMVQm7Din2QT7woVj7CSLOVIA2dubHziT gr4NL1JD2oKEm1+KynWmZgRhW0U44EU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-Ryg-fxAdMkms-vm90M751Q-1; Tue, 26 Jan 2021 06:57:11 -0500 X-MC-Unique: Ryg-fxAdMkms-vm90M751Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9F94E107ACF6; Tue, 26 Jan 2021 11:57:06 +0000 (UTC) Received: from [10.36.114.192] (ovpn-114-192.ams2.redhat.com [10.36.114.192]) by smtp.corp.redhat.com (Postfix) with ESMTP id B16B65D751; Tue, 26 Jan 2021 11:56:49 +0000 (UTC) To: Michal Hocko , Mike Rapoport References: <20210121122723.3446-1-rppt@kernel.org> <20210121122723.3446-8-rppt@kernel.org> <20210126114657.GL827@dhcp22.suse.cz> From: David Hildenbrand Organization: Red Hat GmbH Subject: Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation Message-ID: <303f348d-e494-e386-d1f5-14505b5da254@redhat.com> Date: Tue, 26 Jan 2021 12:56:48 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <20210126114657.GL827@dhcp22.suse.cz> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Message-ID-Hash: P5Y3CLMDHMZHQKUTXPBSDTZYJYDJ2RB2 X-Message-ID-Hash: P5Y3CLMDHMZHQKUTXPBSDTZYJYDJ2RB2 X-MailFrom: david@redhat.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Andrew Morton , Alexander Viro , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Christopher Lameter , Dave Hansen , Elena Reshetova , "H. Peter Anvin" , Ingo Molnar , James Bottomley , "Kirill A. Shutemov" , Matthew Wilcox , Mark Rutland , Mike Rapoport , Michael Kerrisk , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Rick Edgecombe , Roman Gushchin , Shakeel Butt , Shuah Khan , Thomas Gleixner , Tycho Andersen , Will Deacon , linux-api@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-nvdimm@lists.01.org, linux-riscv@lists.infradead.org, x86@kernel.org, Hagen Paul Pfeifer , Palmer Dabbelt X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii"; format="flowed" Content-Transfer-Encoding: 7bit On 26.01.21 12:46, Michal Hocko wrote: > On Thu 21-01-21 14:27:19, Mike Rapoport wrote: >> From: Mike Rapoport >> >> Removing a PAGE_SIZE page from the direct map every time such page is >> allocated for a secret memory mapping will cause severe fragmentation of >> the direct map. This fragmentation can be reduced by using PMD-size pages >> as a pool for small pages for secret memory mappings. >> >> Add a gen_pool per secretmem inode and lazily populate this pool with >> PMD-size pages. >> >> As pages allocated by secretmem become unmovable, use CMA to back large >> page caches so that page allocator won't be surprised by failing attempt to >> migrate these pages. >> >> The CMA area used by secretmem is controlled by the "secretmem=" kernel >> parameter. This allows explicit control over the memory available for >> secretmem and provides upper hard limit for secretmem consumption. > > OK, so I have finally had a look at this closer and this is really not > acceptable. I have already mentioned that in a response to other patch > but any task is able to deprive access to secret memory to other tasks > and cause OOM killer which wouldn't really recover ever and potentially > panic the system. Now you could be less drastic and only make SIGBUS on > fault but that would be still quite terrible. There is a very good > reason why hugetlb implements is non-trivial reservation system to avoid > exactly these problems. > > So unless I am really misreading the code > Nacked-by: Michal Hocko > > That doesn't mean I reject the whole idea. There are some details to > sort out as mentioned elsewhere but you cannot really depend on > pre-allocated pool which can fail at a fault time like that. So, to do it similar to hugetlbfs (e.g., with CMA), there would have to be a mechanism to actually try pre-reserving (e.g., from the CMA area), at which point in time the pages would get moved to the secretmem pool, and a mechanism for mmap() etc. to "reserve" from these secretmem pool, such that there are guarantees at fault time? What we have right now feels like some kind of overcommit (reading, as overcommiting huge pages, so we might get SIGBUS at fault time). TBH, the SIGBUS thingy doesn't sound terrible to me - if this behavior is to be expected right now by applications using it and they can handle it - no guarantees. I fully agree that some kind of reservation/guarantee mechanism would be preferable. -- Thanks, David / dhildenb _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C646C433DB for ; Tue, 26 Jan 2021 11:57:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C62362311D for ; Tue, 26 Jan 2021 11:57:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C62362311D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:Subject: From:References:To:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gZx0Y3SzRdLEd3nMtk6tt9IRp9ByVUtltOBriPT3ivs=; b=I6i6VVcZcsYMUkLXcHyprJVKC rRB8rqST6KBAOWLwR9UuziNUWiMKTzA7U0UOeoup0p+Kk/+DkCn+JjrBIrA5vFwg4RZz9kP7rd7db XXCcB2SDlyaSqwd0CmN3J2lUMntkQtmh+BOza3TeDqZUVMRQgW/U1zt0IKFIWWiKnQoh3VeRaU3yW w4qcNTTBtpbpMI1TpX0C4Si0nWFKIvmGUBu2XeKCm3lWq+63NKmsh4hnaN4LR1IDL9BwYa5q6Ccm1 1VToj1ZcWcnvzT2eljXKLkwPHzfxgRKh9UA68YC6mtgZKaOYoWsJG9XmGTence/7KOBXrU0M0lAHi INx4kBZhA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l4MyE-0001iP-PX; Tue, 26 Jan 2021 11:57:18 +0000 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l4MyB-0001gt-Qa for linux-riscv@lists.infradead.org; Tue, 26 Jan 2021 11:57:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611662235; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qjJ7EBIraH0kcBiHWVTNphRxbXFU8GNfvJrsoVivTWk=; b=V8WD8NCMWlFDx9o6YzcO4I4dENQNX15Lz56GUb71wwvfnEUCghoGq2WMCpCus3UksH4ryw zGNU8VB40n7JCwxI/iYewHuZuQskrP2EPqwOLVMVQm7Din2QT7woVj7CSLOVIA2dubHziT gr4NL1JD2oKEm1+KynWmZgRhW0U44EU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-Ryg-fxAdMkms-vm90M751Q-1; Tue, 26 Jan 2021 06:57:11 -0500 X-MC-Unique: Ryg-fxAdMkms-vm90M751Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9F94E107ACF6; Tue, 26 Jan 2021 11:57:06 +0000 (UTC) Received: from [10.36.114.192] (ovpn-114-192.ams2.redhat.com [10.36.114.192]) by smtp.corp.redhat.com (Postfix) with ESMTP id B16B65D751; Tue, 26 Jan 2021 11:56:49 +0000 (UTC) To: Michal Hocko , Mike Rapoport References: <20210121122723.3446-1-rppt@kernel.org> <20210121122723.3446-8-rppt@kernel.org> <20210126114657.GL827@dhcp22.suse.cz> From: David Hildenbrand Organization: Red Hat GmbH Subject: Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation Message-ID: <303f348d-e494-e386-d1f5-14505b5da254@redhat.com> Date: Tue, 26 Jan 2021 12:56:48 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <20210126114657.GL827@dhcp22.suse.cz> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210126_065716_098157_DF46C1D5 X-CRM114-Status: GOOD ( 24.94 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Zijlstra , Catalin Marinas , Dave Hansen , linux-mm@kvack.org, linux-kselftest@vger.kernel.org, "H. Peter Anvin" , Christopher Lameter , Shuah Khan , Thomas Gleixner , Elena Reshetova , linux-arch@vger.kernel.org, Tycho Andersen , linux-nvdimm@lists.01.org, Will Deacon , x86@kernel.org, Matthew Wilcox , Mike Rapoport , Ingo Molnar , Michael Kerrisk , Palmer Dabbelt , Arnd Bergmann , James Bottomley , Hagen Paul Pfeifer , Borislav Petkov , Alexander Viro , Andy Lutomirski , Paul Walmsley , "Kirill A. Shutemov" , Dan Williams , linux-arm-kernel@lists.infradead.org, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Palmer Dabbelt , linux-fsdevel@vger.kernel.org, Shakeel Butt , Andrew Morton , Rick Edgecombe , Roman Gushchin Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On 26.01.21 12:46, Michal Hocko wrote: > On Thu 21-01-21 14:27:19, Mike Rapoport wrote: >> From: Mike Rapoport >> >> Removing a PAGE_SIZE page from the direct map every time such page is >> allocated for a secret memory mapping will cause severe fragmentation of >> the direct map. This fragmentation can be reduced by using PMD-size pages >> as a pool for small pages for secret memory mappings. >> >> Add a gen_pool per secretmem inode and lazily populate this pool with >> PMD-size pages. >> >> As pages allocated by secretmem become unmovable, use CMA to back large >> page caches so that page allocator won't be surprised by failing attempt to >> migrate these pages. >> >> The CMA area used by secretmem is controlled by the "secretmem=" kernel >> parameter. This allows explicit control over the memory available for >> secretmem and provides upper hard limit for secretmem consumption. > > OK, so I have finally had a look at this closer and this is really not > acceptable. I have already mentioned that in a response to other patch > but any task is able to deprive access to secret memory to other tasks > and cause OOM killer which wouldn't really recover ever and potentially > panic the system. Now you could be less drastic and only make SIGBUS on > fault but that would be still quite terrible. There is a very good > reason why hugetlb implements is non-trivial reservation system to avoid > exactly these problems. > > So unless I am really misreading the code > Nacked-by: Michal Hocko > > That doesn't mean I reject the whole idea. There are some details to > sort out as mentioned elsewhere but you cannot really depend on > pre-allocated pool which can fail at a fault time like that. So, to do it similar to hugetlbfs (e.g., with CMA), there would have to be a mechanism to actually try pre-reserving (e.g., from the CMA area), at which point in time the pages would get moved to the secretmem pool, and a mechanism for mmap() etc. to "reserve" from these secretmem pool, such that there are guarantees at fault time? What we have right now feels like some kind of overcommit (reading, as overcommiting huge pages, so we might get SIGBUS at fault time). TBH, the SIGBUS thingy doesn't sound terrible to me - if this behavior is to be expected right now by applications using it and they can handle it - no guarantees. I fully agree that some kind of reservation/guarantee mechanism would be preferable. -- Thanks, David / dhildenb _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE1B2C433DB for ; Tue, 26 Jan 2021 11:58:56 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 97EC82311C for ; Tue, 26 Jan 2021 11:58:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 97EC82311C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:Subject: From:References:To:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=t/W6W9qS5lhjF6+5pYq9lP9Z8WyC7tvGL/jZdA+8Ugg=; b=piFMfn7cCx+0u9mpI5stCA9df +49NUucIDUdLxKY/T8HDmImGlfwskkHUukOqox47QDwrt6gjqpBQE9FwOARYQVCyNd6o9G4s3POII adOv0aSNJCqabIE/tt3pncR+/nVBCLnsJGjAX0/83lwbuzJqnry+HUFApYHcxGjmQoXQNn6w2kg/9 W9aVOqcei1VPoinEPniUx3yMQJ+yDHZk1xKdSh4md7bPdsgQQMp7mOopl77+77dWaipBD8VIf2JOw z2uMWdEqv+RFzWnMBcWRsvPsvJvKFPCCCmD6ZuRFqkw6kY9RnVsJw8O3OOhhv4y4ScJGRnJp98j5P GyyHZv9Tw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l4MyF-0001il-ED; Tue, 26 Jan 2021 11:57:19 +0000 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l4MyB-0001gx-Ql for linux-arm-kernel@lists.infradead.org; Tue, 26 Jan 2021 11:57:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611662235; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qjJ7EBIraH0kcBiHWVTNphRxbXFU8GNfvJrsoVivTWk=; b=V8WD8NCMWlFDx9o6YzcO4I4dENQNX15Lz56GUb71wwvfnEUCghoGq2WMCpCus3UksH4ryw zGNU8VB40n7JCwxI/iYewHuZuQskrP2EPqwOLVMVQm7Din2QT7woVj7CSLOVIA2dubHziT gr4NL1JD2oKEm1+KynWmZgRhW0U44EU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-Ryg-fxAdMkms-vm90M751Q-1; Tue, 26 Jan 2021 06:57:11 -0500 X-MC-Unique: Ryg-fxAdMkms-vm90M751Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9F94E107ACF6; Tue, 26 Jan 2021 11:57:06 +0000 (UTC) Received: from [10.36.114.192] (ovpn-114-192.ams2.redhat.com [10.36.114.192]) by smtp.corp.redhat.com (Postfix) with ESMTP id B16B65D751; Tue, 26 Jan 2021 11:56:49 +0000 (UTC) To: Michal Hocko , Mike Rapoport References: <20210121122723.3446-1-rppt@kernel.org> <20210121122723.3446-8-rppt@kernel.org> <20210126114657.GL827@dhcp22.suse.cz> From: David Hildenbrand Organization: Red Hat GmbH Subject: Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation Message-ID: <303f348d-e494-e386-d1f5-14505b5da254@redhat.com> Date: Tue, 26 Jan 2021 12:56:48 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <20210126114657.GL827@dhcp22.suse.cz> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210126_065716_144086_2FD96DC0 X-CRM114-Status: GOOD ( 25.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Zijlstra , Catalin Marinas , Dave Hansen , linux-mm@kvack.org, linux-kselftest@vger.kernel.org, "H. Peter Anvin" , Christopher Lameter , Shuah Khan , Thomas Gleixner , Elena Reshetova , linux-arch@vger.kernel.org, Tycho Andersen , linux-nvdimm@lists.01.org, Will Deacon , x86@kernel.org, Matthew Wilcox , Mike Rapoport , Ingo Molnar , Michael Kerrisk , Palmer Dabbelt , Arnd Bergmann , James Bottomley , Hagen Paul Pfeifer , Borislav Petkov , Alexander Viro , Andy Lutomirski , Paul Walmsley , "Kirill A. Shutemov" , Dan Williams , linux-arm-kernel@lists.infradead.org, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Palmer Dabbelt , linux-fsdevel@vger.kernel.org, Shakeel Butt , Andrew Morton , Rick Edgecombe , Roman Gushchin Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 26.01.21 12:46, Michal Hocko wrote: > On Thu 21-01-21 14:27:19, Mike Rapoport wrote: >> From: Mike Rapoport >> >> Removing a PAGE_SIZE page from the direct map every time such page is >> allocated for a secret memory mapping will cause severe fragmentation of >> the direct map. This fragmentation can be reduced by using PMD-size pages >> as a pool for small pages for secret memory mappings. >> >> Add a gen_pool per secretmem inode and lazily populate this pool with >> PMD-size pages. >> >> As pages allocated by secretmem become unmovable, use CMA to back large >> page caches so that page allocator won't be surprised by failing attempt to >> migrate these pages. >> >> The CMA area used by secretmem is controlled by the "secretmem=" kernel >> parameter. This allows explicit control over the memory available for >> secretmem and provides upper hard limit for secretmem consumption. > > OK, so I have finally had a look at this closer and this is really not > acceptable. I have already mentioned that in a response to other patch > but any task is able to deprive access to secret memory to other tasks > and cause OOM killer which wouldn't really recover ever and potentially > panic the system. Now you could be less drastic and only make SIGBUS on > fault but that would be still quite terrible. There is a very good > reason why hugetlb implements is non-trivial reservation system to avoid > exactly these problems. > > So unless I am really misreading the code > Nacked-by: Michal Hocko > > That doesn't mean I reject the whole idea. There are some details to > sort out as mentioned elsewhere but you cannot really depend on > pre-allocated pool which can fail at a fault time like that. So, to do it similar to hugetlbfs (e.g., with CMA), there would have to be a mechanism to actually try pre-reserving (e.g., from the CMA area), at which point in time the pages would get moved to the secretmem pool, and a mechanism for mmap() etc. to "reserve" from these secretmem pool, such that there are guarantees at fault time? What we have right now feels like some kind of overcommit (reading, as overcommiting huge pages, so we might get SIGBUS at fault time). TBH, the SIGBUS thingy doesn't sound terrible to me - if this behavior is to be expected right now by applications using it and they can handle it - no guarantees. I fully agree that some kind of reservation/guarantee mechanism would be preferable. -- Thanks, David / dhildenb _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel