[PATCH v2 0/3] f2fs: support encrypted inline data

[PATCH v2 0/3] f2fs: support encrypted inline data

[LiaoYuanhong-vivo]

F2FS currently disables inline data for encrypted regular files because the
inline payload is stored in the inode block and does not go through the
regular bio-based fscrypt path.  This wastes space for small encrypted
files on Android devices using F2FS inlinecrypt.

This series adds an encrypted_inline_data on-disk feature for F2FS.
With this feature enabled, encrypted regular files may keep small contents
in the inode block.  The inline payload is encrypted before being stored in
the inode and decrypted back into page-cache plaintext on read.

The fscrypt changes are scoped to filesystem-managed data-unit crypto.
F2FS first asks fscrypt whether the inode's key/policy supports this path.
It prepares the software transform only when encrypted inline payloads are
read or written.  Raw-key inlinecrypt support is limited to per-mode
policies (DIRECT_KEY, IV_INO_LBLK_64, and IV_INO_LBLK_32).  Per-file
inlinecrypt keys are not supported for encrypted inline data, to avoid
per-file software tfm memory growth.  Hardware-wrapped keys are not
supported for encrypted inline data.

I tested this on an Android F2FS /data device with inlinecrypt.  The
workload created 10000 encrypted files under the same fscrypt policy.

  Size  encrypted inline_data  Inline sample  fs_used_delta_kb  Avg B/file
  1K    enabled                200/200        46344             4745.63
  4K    enabled                0/200          85280             8732.67
  1K    disabled               0/200          88808             9093.94
  4K    disabled               0/200          80728             8266.55

For the 1K workload, encrypted inline data saved 42464 KiB for 10000 files,
about 4348 bytes per file, or a 47.8% reduction in filesystem used space.
A raw inode check of a sampled file confirmed that the inline region did
not contain plaintext.  The 4K control workload did not retain inline data,
as expected.

This is Android-focused, but the use case is meaningful in practice.  Real
phones can have more than 200000 encrypted files smaller than 4K under
/data.  Avoiding one 4K data block for a large fraction of those files can
save several hundred MiB, and in some cases close to 1GiB.

If keeping this limited to raw-key inlinecrypt is not the right tradeoff,
I'd appreciate suggestions on how encrypted inline data could be supported
with hardware-wrapped keys.

Changes in v2:
- Split fscrypt capability checking from software transform preparation.
- Limit raw-key inlinecrypt support to per-mode policies; per-file
  inlinecrypt keys and hardware-wrapped keys are unsupported.
- Use one data-unit helper and process inline payloads by fscrypt data-unit
  size.
- Update F2FS inline-data paths and documentation.

LiaoYuanhong-vivo (3):
  fscrypt: prepare software keys for filesystem-managed data units
  f2fs: support encrypted inline data
  Documentation: f2fs: document encrypted inline data

 Documentation/ABI/testing/sysfs-fs-f2fs |   5 +-
 Documentation/filesystems/f2fs.rst      |  34 +++++
 fs/crypto/crypto.c                      |  43 ++++++
 fs/crypto/fscrypt_private.h             |   3 +-
 fs/crypto/keysetup.c                    | 167 ++++++++++++++++++++++++
 fs/f2fs/Kconfig                         |  14 ++
 fs/f2fs/data.c                          |   8 +-
 fs/f2fs/f2fs.h                          |  37 +++++-
 fs/f2fs/file.c                          |  24 +++-
 fs/f2fs/inline.c                        | 131 +++++++++++++++++--
 fs/f2fs/super.c                         |  12 ++
 fs/f2fs/sysfs.c                         |   8 ++
 include/linux/fscrypt.h                 |  24 ++++
 13 files changed, 487 insertions(+), 23 deletions(-)

-- 
2.34.1

[PATCH v2 1/3] fscrypt: prepare software keys for filesystem-managed data units

[LiaoYuanhong-vivo]

Some filesystems store small data regions inside filesystem metadata rather
than submitting them through the normal bio path.  F2FS inline data is one
such case.  When encrypted file contents normally use blk-crypto, these
filesystem-managed regions still need software fscrypt handling because no
data bio is submitted for them.

Add helpers for this case.  fscrypt_supports_data_unit_inplace() lets a
filesystem check whether an inode's current key and policy can support this
path without preparing a software transform.
fscrypt_prepare_data_unit_inplace() prepares the transform when the
filesystem actually needs to read or write such a region.
fscrypt_crypt_data_unit_inplace() encrypts or decrypts an in-page region by
fscrypt data-unit size.

For raw-key inlinecrypt, support is limited to per-mode key policies
(DIRECT_KEY, IV_INO_LBLK_64, and IV_INO_LBLK_32), so the software transform
is shared per mode instead of being allocated per file.  Hardware-wrapped
keys and per-file inlinecrypt keys are not supported for this path.

Signed-off-by: LiaoYuanhong-vivo <liaoyuanhong@vivo.com>
---
Changes in v2:
- Split capability checking from software transform preparation.
- Limit raw-key inlinecrypt support to per-mode policies; reject per-file
  and hardware-wrapped keys.
- Use one direction-aware helper and process regions by fscrypt data-unit
  size.

 fs/crypto/crypto.c          |  47 ++++++++++
 fs/crypto/fscrypt_private.h |   3 +-
 fs/crypto/keysetup.c        | 178 ++++++++++++++++++++++++++++++++++++
 include/linux/fscrypt.h     |  24 ++++++
 4 files changed, 251 insertions(+), 1 deletion(-)

diff --git a/fs/crypto/crypto.c b/fs/crypto/crypto.c
index 570a2231c945..a85ea313b0d9 100644
--- a/fs/crypto/crypto.c
+++ b/fs/crypto/crypto.c
@@ -208,6 +208,53 @@ struct page *fscrypt_encrypt_pagecache_blocks(struct folio *folio,
 }
 EXPORT_SYMBOL(fscrypt_encrypt_pagecache_blocks);
 
+/**
+ * fscrypt_crypt_data_unit_inplace() - En/decrypt data units in-place
+ * @inode:   The inode to which these data units belong
+ * @page:    The page containing the data units to en/decrypt
+ * @len:     Size of the region to en/decrypt.  This must be a multiple of
+ *	     FSCRYPT_CONTENTS_ALIGNMENT.
+ * @offs:    Byte offset within @page at which the region begins
+ * @index:   Fscrypt data unit index of the start of the region
+ * @encrypt: True to encrypt, false to decrypt
+ *
+ * Return: 0 on success; -errno on failure
+ */
+int fscrypt_crypt_data_unit_inplace(const struct inode *inode,
+				    struct page *page, unsigned int len,
+				    unsigned int offs, u64 index, bool encrypt)
+{
+	const struct fscrypt_inode_info *ci = fscrypt_get_inode_info_raw(inode);
+	unsigned int du_size;
+	unsigned int i;
+	int err;
+
+	/*
+	 * Pairs with the smp_store_release() that publishes ->tfm after the
+	 * software transform has been fully initialized.
+	 */
+	if (!ci || !smp_load_acquire(&ci->ci_enc_key.tfm))
+		return -EOPNOTSUPP;
+	du_size = 1U << ci->ci_data_unit_bits;
+
+	if (WARN_ON_ONCE(len <= 0))
+		return -EINVAL;
+	if (WARN_ON_ONCE(!IS_ALIGNED(len | offs, FSCRYPT_CONTENTS_ALIGNMENT)))
+		return -EINVAL;
+
+	for (i = 0; i < len; i += du_size, index++) {
+		unsigned int todo = min(du_size, len - i);
+
+		err = fscrypt_crypt_data_unit(ci,
+				encrypt ? FS_ENCRYPT : FS_DECRYPT,
+				index, page, page, todo, offs + i);
+		if (err)
+			return err;
+	}
+	return 0;
+}
+EXPORT_SYMBOL(fscrypt_crypt_data_unit_inplace);
+
 /**
  * fscrypt_encrypt_block_inplace() - Encrypt a filesystem block in-place
  * @inode:     The inode to which this block belongs
diff --git a/fs/crypto/fscrypt_private.h b/fs/crypto/fscrypt_private.h
index 8d3c278a7591..b5c0b881fd4b 100644
--- a/fs/crypto/fscrypt_private.h
+++ b/fs/crypto/fscrypt_private.h
@@ -236,7 +236,8 @@ struct fscrypt_symlink_data {
  * @tfm: crypto API transform object
  * @blk_key: key for blk-crypto
  *
- * Normally only one of the fields will be non-NULL.
+ * Most users need only one prepared form.  Inline-crypto users that also need
+ * filesystem-layer software crypto for non-bio data regions may prepare both.
  */
 struct fscrypt_prepared_key {
 	struct crypto_sync_skcipher *tfm;
diff --git a/fs/crypto/keysetup.c b/fs/crypto/keysetup.c
index ce327bfdada4..2cb629d4383c 100644
--- a/fs/crypto/keysetup.c
+++ b/fs/crypto/keysetup.c
@@ -400,6 +400,184 @@ static int fscrypt_setup_v2_file_key(struct fscrypt_inode_info *ci,
 	return 0;
 }
 
+static int fscrypt_prepare_data_unit_inplace_software_key(
+					struct fscrypt_prepared_key *prep_key,
+					const u8 *raw_key,
+					const struct fscrypt_inode_info *ci)
+{
+	struct crypto_sync_skcipher *tfm;
+
+	/* Pairs with the smp_store_release() below. */
+	if (smp_load_acquire(&prep_key->tfm))
+		return 0;
+	tfm = fscrypt_allocate_skcipher(ci->ci_mode, raw_key, ci->ci_inode);
+	if (IS_ERR(tfm))
+		return PTR_ERR(tfm);
+	/* Pairs with the smp_load_acquire() above and other ->tfm readers. */
+	smp_store_release(&prep_key->tfm, tfm);
+	return 0;
+}
+
+static int fscrypt_prepare_per_mode_data_unit_inplace_key(
+					struct fscrypt_inode_info *ci,
+					struct fscrypt_master_key *mk,
+					struct fscrypt_prepared_key *keys,
+					u8 hkdf_context, bool include_fs_uuid)
+{
+	const struct super_block *sb = ci->ci_inode->i_sb;
+	struct fscrypt_mode *mode = ci->ci_mode;
+	const u8 mode_num = mode - fscrypt_modes;
+	struct fscrypt_prepared_key *prep_key;
+	u8 mode_key[FSCRYPT_MAX_RAW_KEY_SIZE];
+	u8 hkdf_info[sizeof(mode_num) + sizeof(sb->s_uuid)];
+	unsigned int hkdf_infolen = 0;
+	int err;
+
+	if (WARN_ON_ONCE(mode_num > FSCRYPT_MODE_MAX))
+		return -EINVAL;
+
+	prep_key = &keys[mode_num];
+
+	BUILD_BUG_ON(sizeof(mode_num) != 1);
+	BUILD_BUG_ON(sizeof(sb->s_uuid) != 16);
+	BUILD_BUG_ON(sizeof(hkdf_info) != 17);
+	hkdf_info[hkdf_infolen++] = mode_num;
+	if (include_fs_uuid) {
+		memcpy(&hkdf_info[hkdf_infolen], &sb->s_uuid,
+		       sizeof(sb->s_uuid));
+		hkdf_infolen += sizeof(sb->s_uuid);
+	}
+
+	fscrypt_hkdf_expand(&mk->mk_secret.hkdf, hkdf_context, hkdf_info,
+			    hkdf_infolen, mode_key, mode->keysize);
+	err = fscrypt_prepare_data_unit_inplace_software_key(prep_key,
+							    mode_key, ci);
+	memzero_explicit(mode_key, mode->keysize);
+	if (!err)
+		ci->ci_enc_key = *prep_key;
+	return err;
+}
+
+/**
+ * fscrypt_supports_data_unit_inplace() - check data-unit crypto support
+ * @inode: an encrypted regular file inode
+ *
+ * Check whether filesystem-managed data regions can use fscrypt contents
+ * encryption for this inode.  Per-file inlinecrypt keys are intentionally
+ * unsupported to avoid per-file software tfm memory growth.  Hardware-wrapped
+ * keys are unsupported because the software contents key is not available.
+ */
+bool fscrypt_supports_data_unit_inplace(const struct inode *inode)
+{
+	struct fscrypt_inode_info *ci = fscrypt_get_inode_info_raw(inode);
+	struct fscrypt_master_key *mk;
+	u8 flags;
+	bool supported;
+
+	if (!ci)
+		return false;
+	/*
+	 * Pairs with the smp_store_release() that publishes ->tfm after the
+	 * software transform has been fully initialized.
+	 */
+	if (smp_load_acquire(&ci->ci_enc_key.tfm))
+		return true;
+	if (!fscrypt_using_inline_encryption(ci))
+		return false;
+
+	mk = ci->ci_master_key;
+	if (!mk)
+		return false;
+
+	down_read(&mk->mk_sem);
+	if (!mk->mk_present || mk->mk_secret.is_hw_wrapped ||
+	    ci->ci_policy.version != FSCRYPT_POLICY_V2) {
+		supported = false;
+		goto out;
+	}
+
+	flags = ci->ci_policy.v2.flags;
+	supported = flags & (FSCRYPT_POLICY_FLAG_DIRECT_KEY |
+			     FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64 |
+			     FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32);
+out:
+	up_read(&mk->mk_sem);
+	return supported;
+}
+EXPORT_SYMBOL_GPL(fscrypt_supports_data_unit_inplace);
+
+/**
+ * fscrypt_prepare_data_unit_inplace() - prepare software data-unit crypto
+ * @inode: an encrypted regular file inode
+ *
+ * Prepare the software transform used by filesystem-managed data regions that
+ * need fscrypt contents encryption but do not go through a data bio.  If the
+ * inode already uses filesystem-layer encryption, the normal contents key is
+ * already prepared.  If the inode uses blk-crypto with a raw per-mode key, this
+ * prepares the software form of that same per-mode contents key.
+ */
+int fscrypt_prepare_data_unit_inplace(const struct inode *inode)
+{
+	struct fscrypt_inode_info *ci = fscrypt_get_inode_info_raw(inode);
+	struct fscrypt_master_key *mk;
+	u8 flags;
+	int err = 0;
+
+	if (!ci)
+		return -ENOKEY;
+	/*
+	 * Pairs with the smp_store_release() that publishes ->tfm after the
+	 * software transform has been fully initialized.
+	 */
+	if (smp_load_acquire(&ci->ci_enc_key.tfm))
+		return 0;
+	if (!fscrypt_using_inline_encryption(ci))
+		return -EOPNOTSUPP;
+
+	mk = ci->ci_master_key;
+	if (!mk)
+		return -EOPNOTSUPP;
+
+	down_read(&mk->mk_sem);
+	if (!mk->mk_present) {
+		err = -ENOKEY;
+		goto out_unlock;
+	}
+	if (mk->mk_secret.is_hw_wrapped ||
+	    ci->ci_policy.version != FSCRYPT_POLICY_V2) {
+		err = -EOPNOTSUPP;
+		goto out_unlock;
+	}
+
+	mutex_lock(&fscrypt_mode_key_setup_mutex);
+	/* Pairs with fscrypt_prepare_data_unit_inplace_software_key(). */
+	if (smp_load_acquire(&ci->ci_enc_key.tfm))
+		goto out_mutex;
+
+	flags = ci->ci_policy.v2.flags;
+	if (flags & FSCRYPT_POLICY_FLAG_DIRECT_KEY) {
+		err = fscrypt_prepare_per_mode_data_unit_inplace_key(ci, mk,
+				mk->mk_direct_keys, HKDF_CONTEXT_DIRECT_KEY,
+				false);
+	} else if (flags & FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64) {
+		err = fscrypt_prepare_per_mode_data_unit_inplace_key(ci, mk,
+				mk->mk_iv_ino_lblk_64_keys,
+				HKDF_CONTEXT_IV_INO_LBLK_64_KEY, true);
+	} else if (flags & FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32) {
+		err = fscrypt_prepare_per_mode_data_unit_inplace_key(ci, mk,
+				mk->mk_iv_ino_lblk_32_keys,
+				HKDF_CONTEXT_IV_INO_LBLK_32_KEY, true);
+	} else {
+		err = -EOPNOTSUPP;
+	}
+out_mutex:
+	mutex_unlock(&fscrypt_mode_key_setup_mutex);
+out_unlock:
+	up_read(&mk->mk_sem);
+	return err;
+}
+EXPORT_SYMBOL_GPL(fscrypt_prepare_data_unit_inplace);
+
 /*
  * Check whether the size of the given master key (@mk) is appropriate for the
  * encryption settings which a particular file will use (@ci).
diff --git a/include/linux/fscrypt.h b/include/linux/fscrypt.h
index 54712ec61ffb..2702bf1018c1 100644
--- a/include/linux/fscrypt.h
+++ b/include/linux/fscrypt.h
@@ -346,6 +346,12 @@ void fscrypt_enqueue_decrypt_work(struct work_struct *);
 
 struct page *fscrypt_encrypt_pagecache_blocks(struct folio *folio,
 		size_t len, size_t offs, gfp_t gfp_flags);
+
+bool fscrypt_supports_data_unit_inplace(const struct inode *inode);
+int fscrypt_prepare_data_unit_inplace(const struct inode *inode);
+int fscrypt_crypt_data_unit_inplace(const struct inode *inode,
+				    struct page *page, unsigned int len,
+				    unsigned int offs, u64 index, bool encrypt);
 int fscrypt_encrypt_block_inplace(const struct inode *inode, struct page *page,
 				  unsigned int len, unsigned int offs,
 				  u64 lblk_num);
@@ -519,6 +525,24 @@ static inline struct page *fscrypt_encrypt_pagecache_blocks(struct folio *folio,
 	return ERR_PTR(-EOPNOTSUPP);
 }
 
+static inline bool fscrypt_supports_data_unit_inplace(const struct inode *inode)
+{
+	return false;
+}
+
+static inline int fscrypt_prepare_data_unit_inplace(const struct inode *inode)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int fscrypt_crypt_data_unit_inplace(const struct inode *inode,
+						  struct page *page, unsigned int len,
+						  unsigned int offs, u64 index,
+						  bool encrypt)
+{
+	return -EOPNOTSUPP;
+}
+
 static inline int fscrypt_encrypt_block_inplace(const struct inode *inode,
 						struct page *page,
 						unsigned int len,
-- 
2.34.1

Re: [PATCH v2 0/3] f2fs: support encrypted inline data

[LiaoYuanhong-vivo]

Hi,

Gentle ping on this series.

v2 tries to address the previous concerns by avoiding per-file software
tfm growth, preparing the software transform lazily, and explicitly
disabling unsupported key combinations.

The main remaining limitation is hardware-wrapped keys. If this makes
the feature unlikely to be accepted, please let me know. Otherwise, I
would appreciate any review comments on the current direction.

If maintainers have any feasible direction in mind, I would also
appreciate hearing it.

Thanks,
Liao Yuanhong

Re: [PATCH v2 0/3] f2fs: support encrypted inline data

[Jaegeuk Kim]

On 06/11, LiaoYuanhong-vivo wrote:
> Hi,
> 
> Gentle ping on this series.
> 
> v2 tries to address the previous concerns by avoiding per-file software
> tfm growth, preparing the software transform lazily, and explicitly
> disabling unsupported key combinations.
> 
> The main remaining limitation is hardware-wrapped keys. If this makes
> the feature unlikely to be accepted, please let me know. Otherwise, I
> would appreciate any review comments on the current direction.

Yeah, that'd be a big win, if we have the hardware-wrapped key support. By
any chance, can you add it in the patch set?

> 
> If maintainers have any feasible direction in mind, I would also
> appreciate hearing it.
> 
> Thanks,
> Liao Yuanhong