From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A9ECC282C2 for ; Wed, 13 Feb 2019 03:31:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E1B5F21904 for ; Wed, 13 Feb 2019 03:31:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=intel-com.20150623.gappssmtp.com header.i=@intel-com.20150623.gappssmtp.com header.b="qnhqPE73" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729890AbfBMDbu (ORCPT ); Tue, 12 Feb 2019 22:31:50 -0500 Received: from mail-ot1-f65.google.com ([209.85.210.65]:46096 "EHLO mail-ot1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729307AbfBMDbu (ORCPT ); Tue, 12 Feb 2019 22:31:50 -0500 Received: by mail-ot1-f65.google.com with SMTP id w25so1560992otm.13 for ; Tue, 12 Feb 2019 19:31:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=KF0zgxPuyMfvXHcwMoxObnPHMJ+VQQ4AjaJw9kOiNvo=; b=qnhqPE73D7EKnK3xRjX5Zvl3kTQQOQ7AvYA7A4VwPEojwx7pApENepji5qc/kYGtac TaX1JCg0KIHCIvkCJusreXXtGZbf4KhirV7Jf+7Q2Rb+Eb1rVUg7euNaUp8lFh6qPMQN lsS/WSJWxu91IOaylEOibK+wU0L/RtnlffWt0p/c8nNTnk1bmgtMnWPgo2LR4HFaepYy 8dPOrM08opIDn1NhKEL+085jvE6b8+daq7Q9cdK5MTR3WdrffcSmB0yfXftY/vO6Cmoe dqtTcM0pmaA4j9QFqfw05vEa08iX1zWTe7PoYb+r1eIRj1iowa6IdQ5WH8aX8yIw4Cln vtLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=KF0zgxPuyMfvXHcwMoxObnPHMJ+VQQ4AjaJw9kOiNvo=; b=JbWvn0WDSVNVJwSYbvEm2BmKSBoyKU8MLwzsT1LexeIL5esUZt6s6iAtRjmn+vRws3 eLCztQfJBE/cav4VWv7+lOW+NxOWoAxbF3QWfrTqruQFSz/giWe96E673DeQr8lsnUfz pigbmeemmph0vlM/UwY/LVu9Q94bCfiuBiivVjGJAocmVXeq/ilAMstSvxwDN46NfWmW poUGkTHcVd3jjWy43COtwZ/iJ1z1gQfSQpXRDjHAFdfSWIX2/53OytO7rEVKvis/d4ed IXC70in4mh6b3sC3oPtWlAxX50QluKAd+0FENLjrSgbRrhq2HSO2K8WpC3SnXMQe4ES/ GDjA== X-Gm-Message-State: AHQUAuZnw14l8HLm+keM6j3D9NCI7p7SI1GinmrUhE24la1FDWxhK3rK Fm8fGyQjcaXt5YyE1KdKzFmizwub98ULYSryW5wk8A== X-Google-Smtp-Source: AHgI3IartJHr+9QAjB9CRlG6y5nAhyfRmv+egqVU7TNj+GngkKufiwO8grNqF5RgvoZplRr3gUShKkiB/1F8bmHWd4Q= X-Received: by 2002:a9d:7493:: with SMTP id t19mr6850371otk.98.1550028708919; Tue, 12 Feb 2019 19:31:48 -0800 (PST) MIME-Version: 1.0 References: <788d7050-f6bb-b984-69d9-504056e6c5a6@intel.com> <20190212235114.GM20493@dastard> <20190213021318.GN20493@dastard> In-Reply-To: <20190213021318.GN20493@dastard> From: Dan Williams Date: Tue, 12 Feb 2019 19:31:37 -0800 Message-ID: Subject: Re: [LSF/MM TOPIC] Memory Encryption on top of filesystems To: Dave Chinner Cc: Dave Hansen , lsf-pc@lists.linux-foundation.org, linux-fsdevel , Linux-MM , "Shutemov, Kirill" , "Schofield, Alison" , "Darrick J. Wong" , Jan Kara , Christoph Hellwig , "Theodore Ts'o" , Jaegeuk Kim Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org [ add Ted and Jaegeuk ] On Tue, Feb 12, 2019 at 6:14 PM Dave Chinner wrote: > > On Tue, Feb 12, 2019 at 04:27:20PM -0800, Dan Williams wrote: > > On Tue, Feb 12, 2019 at 3:51 PM Dave Chinner wrot= e: > > > > > > On Tue, Feb 12, 2019 at 08:55:57AM -0800, Dave Hansen wrote: > > > > Multi-Key Total Memory Encryption (MKTME) [1] is feature of a memor= y > > > > controller that allows memory to be selectively encrypted with > > > > user-controlled key, in hardware, at a very low runtime cost. Howe= ver, > > > > it is implemented using AES-XTS which encrypts each block with a ke= y > > > > that is generated based on the physical address of the data being > > > > encrypted. This has nice security properties, making some replay a= nd > > > > substitution attacks harder, but it means that encrypted data can n= ot be > > > > naively relocated. > > > > > > The subject is "Memory Encryption on top of filesystems", but really > > > what you are talking about is "physical memory encryption /below/ > > > filesystems". > > > > > > i.e. it's encryption of the physical storage the filesystem manages, > > > not encryption within the fileystem (like fscrypt) or or user data > > > on top of the filesystem (ecryptfs or userspace). > > > > > > > Combined with persistent memory, MKTME allows data to be unlocked a= t the > > > > device (DIMM or namespace) level, but left encrypted until it actua= lly > > > > needs to be used. > > > > > > This sounds more like full disk encryption (either in the IO > > > path software by dm-crypt or in hardware itself), where the contents > > > are decrypted/encrypted in the IO path as the data is moved between > > > physical storage and the filesystem's memory (page/buffer caches). > > > > > > Is there any finer granularity than a DIMM or pmem namespace for > > > specifying encrypted regions? Note that filesystems are not aware of > > > the physical layout of the memory address space (i.e. what DIMM > > > corresponds to which sector in the block device), so DIMM-level > > > granularity doesn't seem particularly useful right now.... > > > > > > Also, how many different hardware encryption keys are available for > > > use, and how many separate memory regions can a single key have > > > associated with it? > > > > > > > However, if encrypted data were placed on a > > > > filesystem, it might be in its encrypted state for long periods of = time > > > > and could not be moved by the filesystem during that time. > > > > > > I'm not sure what you mean by "if encrypted data were placed on a > > > filesystem", given that the memory encryption is transparent to the > > > filesystem (i.e. happens in the memory controller on it's way > > > to/from the physical storage). > > > > > > > The =E2=80=9Ceasy=E2=80=9D solution to this is to just require that= the encryption key > > > > be present and programmed into the memory controller before data is > > > > moved. However, this means that filesystems would need to know whe= n a > > > > given block has been encrypted and can not be moved. > > > > > > I'm missing something here - how does the filesystem even get > > > mounted if we haven't unlocked the device the filesystem is stored > > > on? i.e. we need to unlock the entire memory region containing the > > > filesystem so it can read and write it's metadata (which can be > > > randomly spread all over the block device). > > > > > > And if we have to do that to mount the filesystem, then aren't we > > > also unlocking all the same memory regions that contain user data > > > and hence they can be moved? > > > > Yes, and this is the most likely scenario for enabling MKTME with > > persistent memory. The filesystem will not be able to mount until the > > entire physical address range (namespace device) is unlocked, and the > > filesystem is kept unaware of the encryption. One key per namespace > > device. > > > > > At what point do we end up with a filesystem mounted and trying to > > > access a locked memory region? > > > > Another option is to enable encryption to be specified at mmap time > > with the motivation of being able to use the file system for > > provisioning instead of managing multiple namespaces. > > I'm assuming you are talking about DAX here, yes? > > Because fscrypt.... > > > The filesystem > > would need to be careful to use the key for any physical block > > management, and a decision would need to be made about when/whether > > read(2)/write(2) access cipher text . > > ... already handles all this via page cache coherency for > mmap/read/write IO. Oh! /me checks It handles mmap coherency by making the page cache be clear text, but perhaps in the DAX case we can make it be coherent cipher text through both paths. > > The current thinking is that > > this would be too invasive / restrictive for the filesystem, but it's > > otherwise an interesting thought experiment for allowing the > > filesystem to take on more physical-storage allocation > > responsibilities. > > Actually what we want in the filesystem world is /hardware offload/ > abstractions in the filesystems, not "filesystem controls hardware > specific physical storage features" mechanisms. > > i.e. if the filesystem/fscrypt can offload the encryption of the > data to the IO path by passing the fscrypt key/info with the IO, > then it works with everything, not just pmem. > > In the case of pmem+DAX+mmap(), it needs to associate the correct > key with the memory region that is to be encrypted when it is > mmap()d. Then the DAX subsystem can associate the key with the > physical pages that are faulted during DAX access. If it's bio based > IO going to the DAX driver, then the keys should be attached to the > bio.... > > fscrypt encrypt/decrypt is already done at the filesystem/bio > interface layer via bounce buffers - it's not a great stretch to > push this down a layer so that it can be offloaded to the underlying > device if it is hardware encryption capable. fscrypt would really > only be used for key management (like needs work to support > arbitrary hardware keys) and in filesystem metadata encryption (e.g. > filenames) in that case.... Thanks, yes, fscrypt needs a closer look. As far I can see at a quick glance fscrypt has the same physical block inputs for the encryption algorithm as MKTME so it seems it could be crafted as a drop in accelerator for fscrypt for pmem block devices.