[PATCH v2 0/6] Basic inline encryption support for ufs-exynos

[PATCH v2 0/6] Basic inline encryption support for ufs-exynos

[Eric Biggers]

Add support for Flash Memory Protector (FMP), which is the inline
encryption hardware on Exynos and Exynos-based SoCs.

Specifically, add support for the "traditional FMP mode" that works on
many Exynos-based SoCs including gs101.  This is the mode that uses
"software keys" and is compatible with the upstream kernel's existing
inline encryption framework in the block and filesystem layers.  I plan
to add support for the wrapped key support on gs101 at a later time.

Tested on gs101 (specifically Pixel 6) by running the 'encrypt' group of
xfstests on a filesystem mounted with the 'inlinecrypt' mount option.

This patchset applies to v6.10-rc6, and it has no prerequisites that
aren't already upstream.

Changed in v2:
  - Added DATA_UNIT_SIZE macro
  - Changed a comment into kerneldoc
  - Used ARM_SMCCC_CALL_VAL() to define SMC codes
  - Used arm_smccc_smc() directly instead of via a wrapper function

Eric Biggers (6):
  scsi: ufs: core: Add UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE
  scsi: ufs: core: fold ufshcd_clear_keyslot() into its caller
  scsi: ufs: core: Add UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE
  scsi: ufs: core: Add fill_crypto_prdt variant op
  scsi: ufs: core: Add UFSHCD_QUIRK_KEYS_IN_PRDT
  scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

 drivers/ufs/core/ufshcd-crypto.c |  34 +++--
 drivers/ufs/core/ufshcd-crypto.h |  36 +++++
 drivers/ufs/core/ufshcd.c        |   3 +-
 drivers/ufs/host/ufs-exynos.c    | 228 ++++++++++++++++++++++++++++++-
 include/ufs/ufshcd.h             |  28 ++++
 5 files changed, 308 insertions(+), 21 deletions(-)


base-commit: 22a40d14b572deb80c0648557f4bd502d7e83826
-- 
2.45.2

[PATCH v2 1/6] scsi: ufs: core: Add UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Add UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE which lets UFS host drivers
initialize the blk_crypto_profile themselves rather than have it be
initialized by ufshcd-core according to the UFSHCI standard.  This is
needed to support inline encryption on the "Exynos" UFS controller which
has a nonstandard interface.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 drivers/ufs/core/ufshcd-crypto.c | 10 +++++++---
 include/ufs/ufshcd.h             |  9 +++++++++
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/ufs/core/ufshcd-crypto.c b/drivers/ufs/core/ufshcd-crypto.c
index f2c4422cab86..debc925ae439 100644
--- a/drivers/ufs/core/ufshcd-crypto.c
+++ b/drivers/ufs/core/ufshcd-crypto.c
@@ -157,10 +157,13 @@ int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba)
 {
 	int cap_idx;
 	int err = 0;
 	enum blk_crypto_mode_num blk_mode_num;
 
+	if (hba->quirks & UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE)
+		return 0;
+
 	/*
 	 * Don't use crypto if either the hardware doesn't advertise the
 	 * standard crypto capability bit *or* if the vendor specific driver
 	 * hasn't advertised that crypto is supported.
 	 */
@@ -226,13 +229,14 @@ void ufshcd_init_crypto(struct ufs_hba *hba)
 	int slot;
 
 	if (!(hba->caps & UFSHCD_CAP_CRYPTO))
 		return;
 
-	/* Clear all keyslots - the number of keyslots is (CFGC + 1) */
-	for (slot = 0; slot < hba->crypto_capabilities.config_count + 1; slot++)
-		ufshcd_clear_keyslot(hba, slot);
+	/* Clear all keyslots. */
+	for (slot = 0; slot < hba->crypto_profile.num_slots; slot++)
+		hba->crypto_profile.ll_ops.keyslot_evict(&hba->crypto_profile,
+							 NULL, slot);
 }
 
 void ufshcd_crypto_register(struct ufs_hba *hba, struct request_queue *q)
 {
 	if (hba->caps & UFSHCD_CAP_CRYPTO)
diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h
index bad88bd91995..b354a7eee478 100644
--- a/include/ufs/ufshcd.h
+++ b/include/ufs/ufshcd.h
@@ -641,10 +641,19 @@ enum ufshcd_quirks {
 	/*
 	 * Some host does not implement SQ Run Time Command (SQRTC) register
 	 * thus need this quirk to skip related flow.
 	 */
 	UFSHCD_QUIRK_MCQ_BROKEN_RTC			= 1 << 21,
+
+	/*
+	 * This quirk needs to be enabled if the host controller supports inline
+	 * encryption but it needs to initialize the crypto capabilities in a
+	 * nonstandard way and/or needs to override blk_crypto_ll_ops.  If
+	 * enabled, the standard code won't initialize the blk_crypto_profile;
+	 * ufs_hba_variant_ops::init() must do it instead.
+	 */
+	UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE		= 1 << 22,
 };
 
 enum ufshcd_caps {
 	/* Allow dynamic clk gating */
 	UFSHCD_CAP_CLK_GATING				= 1 << 0,
-- 
2.45.2

Re: [PATCH v2 1/6] scsi: ufs: core: Add UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE

[Peter Griffin]

Hi Eric,

On Tue, 2 Jul 2024 at 08:28, Eric Biggers <ebiggers@kernel.org> wrote:
>
> From: Eric Biggers <ebiggers@google.com>
>
> Add UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE which lets UFS host drivers
> initialize the blk_crypto_profile themselves rather than have it be
> initialized by ufshcd-core according to the UFSHCI standard.  This is
> needed to support inline encryption on the "Exynos" UFS controller which
> has a nonstandard interface.
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---

Reviewed-by:  Peter Griffin <peter.griffin@linaro.org>

[..]

regards,

Peter

[PATCH v2 2/6] scsi: ufs: core: fold ufshcd_clear_keyslot() into its caller

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Fold ufshcd_clear_keyslot() into its only remaining caller.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 drivers/ufs/core/ufshcd-crypto.c | 16 +++++-----------
 1 file changed, 5 insertions(+), 11 deletions(-)

diff --git a/drivers/ufs/core/ufshcd-crypto.c b/drivers/ufs/core/ufshcd-crypto.c
index debc925ae439..b4980fd91cee 100644
--- a/drivers/ufs/core/ufshcd-crypto.c
+++ b/drivers/ufs/core/ufshcd-crypto.c
@@ -93,31 +93,25 @@ static int ufshcd_crypto_keyslot_program(struct blk_crypto_profile *profile,
 
 	memzero_explicit(&cfg, sizeof(cfg));
 	return err;
 }
 
-static int ufshcd_clear_keyslot(struct ufs_hba *hba, int slot)
+static int ufshcd_crypto_keyslot_evict(struct blk_crypto_profile *profile,
+				       const struct blk_crypto_key *key,
+				       unsigned int slot)
 {
+	struct ufs_hba *hba =
+		container_of(profile, struct ufs_hba, crypto_profile);
 	/*
 	 * Clear the crypto cfg on the device. Clearing CFGE
 	 * might not be sufficient, so just clear the entire cfg.
 	 */
 	union ufs_crypto_cfg_entry cfg = {};
 
 	return ufshcd_program_key(hba, &cfg, slot);
 }
 
-static int ufshcd_crypto_keyslot_evict(struct blk_crypto_profile *profile,
-				       const struct blk_crypto_key *key,
-				       unsigned int slot)
-{
-	struct ufs_hba *hba =
-		container_of(profile, struct ufs_hba, crypto_profile);
-
-	return ufshcd_clear_keyslot(hba, slot);
-}
-
 bool ufshcd_crypto_enable(struct ufs_hba *hba)
 {
 	if (!(hba->caps & UFSHCD_CAP_CRYPTO))
 		return false;
 
-- 
2.45.2

Re: [PATCH v2 2/6] scsi: ufs: core: fold ufshcd_clear_keyslot() into its caller

[Peter Griffin]

Hi Eric,

On Tue, 2 Jul 2024 at 08:28, Eric Biggers <ebiggers@kernel.org> wrote:
>
> From: Eric Biggers <ebiggers@google.com>
>
> Fold ufshcd_clear_keyslot() into its only remaining caller.
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---

Reviewed-by:  Peter Griffin <peter.griffin@linaro.org>

[..]

regards,

Peter

[PATCH v2 3/6] scsi: ufs: core: Add UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Add UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE which tells the UFS core to not
use the crypto enable bit defined by the UFS specification.  This is
needed to support inline encryption on the "Exynos" UFS controller.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 drivers/ufs/core/ufshcd-crypto.c | 8 ++++++++
 include/ufs/ufshcd.h             | 7 +++++++
 2 files changed, 15 insertions(+)

diff --git a/drivers/ufs/core/ufshcd-crypto.c b/drivers/ufs/core/ufshcd-crypto.c
index b4980fd91cee..a714dad82cd1 100644
--- a/drivers/ufs/core/ufshcd-crypto.c
+++ b/drivers/ufs/core/ufshcd-crypto.c
@@ -108,17 +108,25 @@ static int ufshcd_crypto_keyslot_evict(struct blk_crypto_profile *profile,
 	union ufs_crypto_cfg_entry cfg = {};
 
 	return ufshcd_program_key(hba, &cfg, slot);
 }
 
+/*
+ * Reprogram the keyslots if needed, and return true if CRYPTO_GENERAL_ENABLE
+ * should be used in the host controller initialization sequence.
+ */
 bool ufshcd_crypto_enable(struct ufs_hba *hba)
 {
 	if (!(hba->caps & UFSHCD_CAP_CRYPTO))
 		return false;
 
 	/* Reset might clear all keys, so reprogram all the keys. */
 	blk_crypto_reprogram_all_keys(&hba->crypto_profile);
+
+	if (hba->quirks & UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE)
+		return false;
+
 	return true;
 }
 
 static const struct blk_crypto_ll_ops ufshcd_crypto_ops = {
 	.keyslot_program	= ufshcd_crypto_keyslot_program,
diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h
index b354a7eee478..4b7ad23a4420 100644
--- a/include/ufs/ufshcd.h
+++ b/include/ufs/ufshcd.h
@@ -650,10 +650,17 @@ enum ufshcd_quirks {
 	 * nonstandard way and/or needs to override blk_crypto_ll_ops.  If
 	 * enabled, the standard code won't initialize the blk_crypto_profile;
 	 * ufs_hba_variant_ops::init() must do it instead.
 	 */
 	UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE		= 1 << 22,
+
+	/*
+	 * This quirk needs to be enabled if the host controller supports inline
+	 * encryption but does not support the CRYPTO_GENERAL_ENABLE bit, i.e.
+	 * host controller initialization fails if that bit is set.
+	 */
+	UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE		= 1 << 23,
 };
 
 enum ufshcd_caps {
 	/* Allow dynamic clk gating */
 	UFSHCD_CAP_CLK_GATING				= 1 << 0,
-- 
2.45.2

Re: [PATCH v2 3/6] scsi: ufs: core: Add UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE

[Peter Griffin]

Hi Eric,

On Tue, 2 Jul 2024 at 08:28, Eric Biggers <ebiggers@kernel.org> wrote:
>
> From: Eric Biggers <ebiggers@google.com>
>
> Add UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE which tells the UFS core to not
> use the crypto enable bit defined by the UFS specification.  This is
> needed to support inline encryption on the "Exynos" UFS controller.
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---

Reviewed-by:  Peter Griffin <peter.griffin@linaro.org>

[..]

regards,

Peter

[PATCH v2 4/6] scsi: ufs: core: Add fill_crypto_prdt variant op

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Add a variant op to allow host drivers to initialize nonstandard
crypto-related fields in the PRDT.  This is needed to support inline
encryption on the "Exynos" UFS controller.

Note that this will be used together with the support for overriding the
PRDT entry size that was already added by commit ada1e653a5ea ("scsi:
ufs: core: Allow UFS host drivers to override the sg entry size").

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 drivers/ufs/core/ufshcd-crypto.h | 19 +++++++++++++++++++
 drivers/ufs/core/ufshcd.c        |  2 +-
 include/ufs/ufshcd.h             |  4 ++++
 3 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/drivers/ufs/core/ufshcd-crypto.h b/drivers/ufs/core/ufshcd-crypto.h
index be8596f20ba2..3eb8df42e194 100644
--- a/drivers/ufs/core/ufshcd-crypto.h
+++ b/drivers/ufs/core/ufshcd-crypto.h
@@ -35,10 +35,23 @@ ufshcd_prepare_req_desc_hdr_crypto(struct ufshcd_lrb *lrbp,
 	h->cci = lrbp->crypto_key_slot;
 	h->dunl = cpu_to_le32(lower_32_bits(lrbp->data_unit_num));
 	h->dunu = cpu_to_le32(upper_32_bits(lrbp->data_unit_num));
 }
 
+static inline int ufshcd_crypto_fill_prdt(struct ufs_hba *hba,
+					  struct ufshcd_lrb *lrbp)
+{
+	struct scsi_cmnd *cmd = lrbp->cmd;
+	const struct bio_crypt_ctx *crypt_ctx = scsi_cmd_to_rq(cmd)->crypt_ctx;
+
+	if (crypt_ctx && hba->vops && hba->vops->fill_crypto_prdt)
+		return hba->vops->fill_crypto_prdt(hba, crypt_ctx,
+						   lrbp->ucd_prdt_ptr,
+						   scsi_sg_count(cmd));
+	return 0;
+}
+
 bool ufshcd_crypto_enable(struct ufs_hba *hba);
 
 int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba);
 
 void ufshcd_init_crypto(struct ufs_hba *hba);
@@ -52,10 +65,16 @@ static inline void ufshcd_prepare_lrbp_crypto(struct request *rq,
 
 static inline void
 ufshcd_prepare_req_desc_hdr_crypto(struct ufshcd_lrb *lrbp,
 				   struct request_desc_header *h) { }
 
+static inline int ufshcd_crypto_fill_prdt(struct ufs_hba *hba,
+					  struct ufshcd_lrb *lrbp)
+{
+	return 0;
+}
+
 static inline bool ufshcd_crypto_enable(struct ufs_hba *hba)
 {
 	return false;
 }
 
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 1b65e6ae4137..744af9708e51 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -2634,11 +2634,11 @@ static int ufshcd_map_sg(struct ufs_hba *hba, struct ufshcd_lrb *lrbp)
 	if (sg_segments < 0)
 		return sg_segments;
 
 	ufshcd_sgl_to_prdt(hba, lrbp, sg_segments, scsi_sglist(cmd));
 
-	return 0;
+	return ufshcd_crypto_fill_prdt(hba, lrbp);
 }
 
 /**
  * ufshcd_enable_intr - enable interrupts
  * @hba: per adapter instance
diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h
index 4b7ad23a4420..59aa6c831a41 100644
--- a/include/ufs/ufshcd.h
+++ b/include/ufs/ufshcd.h
@@ -319,10 +319,11 @@ struct ufs_pwr_mode_info {
  * @dbg_register_dump: used to dump controller debug information
  * @phy_initialization: used to initialize phys
  * @device_reset: called to issue a reset pulse on the UFS device
  * @config_scaling_param: called to configure clock scaling parameters
  * @program_key: program or evict an inline encryption key
+ * @fill_crypto_prdt: initialize crypto-related fields in the PRDT
  * @event_notify: called to notify important events
  * @reinit_notify: called to notify reinit of UFSHCD during max gear switch
  * @mcq_config_resource: called to configure MCQ platform resources
  * @get_hba_mac: called to get vendor specific mac value, mandatory for mcq mode
  * @op_runtime_config: called to config Operation and runtime regs Pointers
@@ -363,10 +364,13 @@ struct ufs_hba_variant_ops {
 	void	(*config_scaling_param)(struct ufs_hba *hba,
 				struct devfreq_dev_profile *profile,
 				struct devfreq_simple_ondemand_data *data);
 	int	(*program_key)(struct ufs_hba *hba,
 			       const union ufs_crypto_cfg_entry *cfg, int slot);
+	int	(*fill_crypto_prdt)(struct ufs_hba *hba,
+				    const struct bio_crypt_ctx *crypt_ctx,
+				    void *prdt, unsigned int num_segments);
 	void	(*event_notify)(struct ufs_hba *hba,
 				enum ufs_event_type evt, void *data);
 	void	(*reinit_notify)(struct ufs_hba *);
 	int	(*mcq_config_resource)(struct ufs_hba *hba);
 	int	(*get_hba_mac)(struct ufs_hba *hba);
-- 
2.45.2

Re: [PATCH v2 4/6] scsi: ufs: core: Add fill_crypto_prdt variant op

[Peter Griffin]

Hi Eric,

On Tue, 2 Jul 2024 at 08:28, Eric Biggers <ebiggers@kernel.org> wrote:
>
> From: Eric Biggers <ebiggers@google.com>
>
> Add a variant op to allow host drivers to initialize nonstandard
> crypto-related fields in the PRDT.  This is needed to support inline
> encryption on the "Exynos" UFS controller.
>
> Note that this will be used together with the support for overriding the
> PRDT entry size that was already added by commit ada1e653a5ea ("scsi:
> ufs: core: Allow UFS host drivers to override the sg entry size").
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---

Reviewed-by:  Peter Griffin <peter.griffin@linaro.org>

[..]

regards,

Peter

[PATCH v2 5/6] scsi: ufs: core: Add UFSHCD_QUIRK_KEYS_IN_PRDT

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Since the nonstandard inline encryption support on Exynos SoCs requires
that raw cryptographic keys be copied into the PRDT, it is desirable to
zeroize those keys after each request to keep them from being left in
memory.  Therefore, add a quirk bit that enables the zeroization.

We could instead do the zeroization unconditionally.  However, using a
quirk bit avoids adding the zeroization overhead to standard devices.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 drivers/ufs/core/ufshcd-crypto.h | 17 +++++++++++++++++
 drivers/ufs/core/ufshcd.c        |  1 +
 include/ufs/ufshcd.h             |  8 ++++++++
 3 files changed, 26 insertions(+)

diff --git a/drivers/ufs/core/ufshcd-crypto.h b/drivers/ufs/core/ufshcd-crypto.h
index 3eb8df42e194..89bb97c14c15 100644
--- a/drivers/ufs/core/ufshcd-crypto.h
+++ b/drivers/ufs/core/ufshcd-crypto.h
@@ -48,10 +48,24 @@ static inline int ufshcd_crypto_fill_prdt(struct ufs_hba *hba,
 						   lrbp->ucd_prdt_ptr,
 						   scsi_sg_count(cmd));
 	return 0;
 }
 
+static inline void ufshcd_crypto_clear_prdt(struct ufs_hba *hba,
+					    struct ufshcd_lrb *lrbp)
+{
+	if (!(hba->quirks & UFSHCD_QUIRK_KEYS_IN_PRDT))
+		return;
+
+	if (!(scsi_cmd_to_rq(lrbp->cmd)->crypt_ctx))
+		return;
+
+	/* Zeroize the PRDT because it can contain cryptographic keys. */
+	memzero_explicit(lrbp->ucd_prdt_ptr,
+			 ufshcd_sg_entry_size(hba) * scsi_sg_count(lrbp->cmd));
+}
+
 bool ufshcd_crypto_enable(struct ufs_hba *hba);
 
 int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba);
 
 void ufshcd_init_crypto(struct ufs_hba *hba);
@@ -71,10 +85,13 @@ static inline int ufshcd_crypto_fill_prdt(struct ufs_hba *hba,
 					  struct ufshcd_lrb *lrbp)
 {
 	return 0;
 }
 
+static inline void ufshcd_crypto_clear_prdt(struct ufs_hba *hba,
+					    struct ufshcd_lrb *lrbp) { }
+
 static inline bool ufshcd_crypto_enable(struct ufs_hba *hba)
 {
 	return false;
 }
 
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 744af9708e51..958cc73d8e79 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -5472,10 +5472,11 @@ void ufshcd_release_scsi_cmd(struct ufs_hba *hba,
 			     struct ufshcd_lrb *lrbp)
 {
 	struct scsi_cmnd *cmd = lrbp->cmd;
 
 	scsi_dma_unmap(cmd);
+	ufshcd_crypto_clear_prdt(hba, lrbp);
 	ufshcd_release(hba);
 	ufshcd_clk_scaling_update_busy(hba);
 }
 
 /**
diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h
index 59aa6c831a41..fe0073b37224 100644
--- a/include/ufs/ufshcd.h
+++ b/include/ufs/ufshcd.h
@@ -661,10 +661,18 @@ enum ufshcd_quirks {
 	 * This quirk needs to be enabled if the host controller supports inline
 	 * encryption but does not support the CRYPTO_GENERAL_ENABLE bit, i.e.
 	 * host controller initialization fails if that bit is set.
 	 */
 	UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE		= 1 << 23,
+
+	/*
+	 * This quirk needs to be enabled if the host controller driver copies
+	 * cryptographic keys into the PRDT in order to send them to hardware,
+	 * and therefore the PRDT should be zeroized after each request (as per
+	 * the standard best practice for managing keys).
+	 */
+	UFSHCD_QUIRK_KEYS_IN_PRDT			= 1 << 24,
 };
 
 enum ufshcd_caps {
 	/* Allow dynamic clk gating */
 	UFSHCD_CAP_CLK_GATING				= 1 << 0,
-- 
2.45.2

Re: [PATCH v2 5/6] scsi: ufs: core: Add UFSHCD_QUIRK_KEYS_IN_PRDT

[Peter Griffin]

Hi Eric,

On Tue, 2 Jul 2024 at 08:28, Eric Biggers <ebiggers@kernel.org> wrote:
>
> From: Eric Biggers <ebiggers@google.com>
>
> Since the nonstandard inline encryption support on Exynos SoCs requires
> that raw cryptographic keys be copied into the PRDT, it is desirable to
> zeroize those keys after each request to keep them from being left in
> memory.  Therefore, add a quirk bit that enables the zeroization.
>
> We could instead do the zeroization unconditionally.  However, using a
> quirk bit avoids adding the zeroization overhead to standard devices.
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---

Reviewed-by:  Peter Griffin <peter.griffin@linaro.org>

[..]

regards,

Peter

[PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Add support for Flash Memory Protector (FMP), which is the inline
encryption hardware on Exynos and Exynos-based SoCs.

Specifically, add support for the "traditional FMP mode" that works on
many Exynos-based SoCs including gs101.  This is the mode that uses
"software keys" and is compatible with the upstream kernel's existing
inline encryption framework in the block and filesystem layers.  I plan
to add support for the wrapped key support on gs101 at a later time.

Tested on gs101 (specifically Pixel 6) by running the 'encrypt' group of
xfstests on a filesystem mounted with the 'inlinecrypt' mount option.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 drivers/ufs/host/ufs-exynos.c | 228 +++++++++++++++++++++++++++++++++-
 1 file changed, 222 insertions(+), 6 deletions(-)

diff --git a/drivers/ufs/host/ufs-exynos.c b/drivers/ufs/host/ufs-exynos.c
index 88d125d1ee3c..dd545ef7c361 100644
--- a/drivers/ufs/host/ufs-exynos.c
+++ b/drivers/ufs/host/ufs-exynos.c
@@ -6,10 +6,13 @@
  * Author: Seungwon Jeon  <essuuj@gmail.com>
  * Author: Alim Akhtar <alim.akhtar@samsung.com>
  *
  */
 
+#include <asm/unaligned.h>
+#include <crypto/aes.h>
+#include <linux/arm-smccc.h>
 #include <linux/clk.h>
 #include <linux/delay.h>
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/of_address.h>
@@ -23,16 +26,18 @@
 #include <ufs/ufshci.h>
 #include <ufs/unipro.h>
 
 #include "ufs-exynos.h"
 
+#define DATA_UNIT_SIZE		4096
+#define LOG2_DATA_UNIT_SIZE	12
+
 /*
  * Exynos's Vendor specific registers for UFSHCI
  */
 #define HCI_TXPRDT_ENTRY_SIZE	0x00
 #define PRDT_PREFECT_EN		BIT(31)
-#define PRDT_SET_SIZE(x)	((x) & 0x1F)
 #define HCI_RXPRDT_ENTRY_SIZE	0x04
 #define HCI_1US_TO_CNT_VAL	0x0C
 #define CNT_VAL_1US_MASK	0x3FF
 #define HCI_UTRL_NEXUS_TYPE	0x40
 #define HCI_UTMRL_NEXUS_TYPE	0x44
@@ -1041,12 +1046,12 @@ static int exynos_ufs_post_link(struct ufs_hba *hba)
 
 	exynos_ufs_establish_connt(ufs);
 	exynos_ufs_fit_aggr_timeout(ufs);
 
 	hci_writel(ufs, 0xa, HCI_DATA_REORDER);
-	hci_writel(ufs, PRDT_SET_SIZE(12), HCI_TXPRDT_ENTRY_SIZE);
-	hci_writel(ufs, PRDT_SET_SIZE(12), HCI_RXPRDT_ENTRY_SIZE);
+	hci_writel(ufs, LOG2_DATA_UNIT_SIZE, HCI_TXPRDT_ENTRY_SIZE);
+	hci_writel(ufs, LOG2_DATA_UNIT_SIZE, HCI_RXPRDT_ENTRY_SIZE);
 	hci_writel(ufs, (1 << hba->nutrs) - 1, HCI_UTRL_NEXUS_TYPE);
 	hci_writel(ufs, (1 << hba->nutmrs) - 1, HCI_UTMRL_NEXUS_TYPE);
 	hci_writel(ufs, 0xf, HCI_AXIDMA_RWDATA_BURST_LEN);
 
 	if (ufs->opts & EXYNOS_UFS_OPT_SKIP_CONNECTION_ESTAB)
@@ -1149,10 +1154,218 @@ static inline void exynos_ufs_priv_init(struct ufs_hba *hba,
 		ufs->rx_sel_idx = 0;
 	hba->priv = (void *)ufs;
 	hba->quirks = ufs->drv_data->quirks;
 }
 
+#ifdef CONFIG_SCSI_UFS_CRYPTO
+
+/*
+ * Support for Flash Memory Protector (FMP), which is the inline encryption
+ * hardware on Exynos and Exynos-based SoCs.  The interface to this hardware is
+ * not compatible with the standard UFS crypto.  It requires that encryption be
+ * configured in the PRDT using a nonstandard extension.
+ */
+
+enum fmp_crypto_algo_mode {
+	FMP_BYPASS_MODE = 0,
+	FMP_ALGO_MODE_AES_CBC = 1,
+	FMP_ALGO_MODE_AES_XTS = 2,
+};
+enum fmp_crypto_key_length {
+	FMP_KEYLEN_256BIT = 1,
+};
+
+/**
+ * struct fmp_sg_entry - nonstandard format of PRDT entries when FMP is enabled
+ *
+ * @base: The standard PRDT entry, but with nonstandard bitfields in the high
+ *	bits of the 'size' field, i.e. the last 32-bit word.  When these
+ *	nonstandard bitfields are zero, the data segment won't be encrypted or
+ *	decrypted.  Otherwise they specify the algorithm and key length with
+ *	which the data segment will be encrypted or decrypted.
+ * @file_iv: The initialization vector (IV) with all bytes reversed
+ * @file_enckey: The first half of the AES-XTS key with all bytes reserved
+ * @file_twkey: The second half of the AES-XTS key with all bytes reserved
+ * @disk_iv: Unused
+ * @reserved: Unused
+ */
+struct fmp_sg_entry {
+	struct ufshcd_sg_entry base;
+	__be64 file_iv[2];
+	__be64 file_enckey[4];
+	__be64 file_twkey[4];
+	__be64 disk_iv[2];
+	__be64 reserved[2];
+};
+
+#define SMC_CMD_FMP_SECURITY	\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
+			   ARM_SMCCC_OWNER_SIP, 0x1810)
+#define SMC_CMD_SMU		\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
+			   ARM_SMCCC_OWNER_SIP, 0x1850)
+#define SMC_CMD_FMP_SMU_RESUME	\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
+			   ARM_SMCCC_OWNER_SIP, 0x1860)
+#define SMU_EMBEDDED			0
+#define SMU_INIT			0
+#define CFG_DESCTYPE_3			3
+
+static void exynos_ufs_fmp_init(struct ufs_hba *hba)
+{
+	struct blk_crypto_profile *profile = &hba->crypto_profile;
+	struct arm_smccc_res res;
+	int err;
+
+	/*
+	 * Check for the standard crypto support bit, since it's available even
+	 * though the rest of the interface to FMP is nonstandard.
+	 *
+	 * This check should have the effect of preventing the driver from
+	 * trying to use FMP on old Exynos SoCs that don't have FMP.
+	 */
+	if (!(ufshcd_readl(hba, REG_CONTROLLER_CAPABILITIES) &
+	      MASK_CRYPTO_SUPPORT))
+		return;
+
+	/*
+	 * This call (which sets DESCTYPE to 0x3 in the FMPSECURITY0 register)
+	 * is needed to make the hardware use the larger PRDT entry size.
+	 */
+	BUILD_BUG_ON(sizeof(struct fmp_sg_entry) != 128);
+	arm_smccc_smc(SMC_CMD_FMP_SECURITY, 0, SMU_EMBEDDED, CFG_DESCTYPE_3,
+		      0, 0, 0, 0, &res);
+	if (res.a0) {
+		dev_warn(hba->dev,
+			 "SMC_CMD_FMP_SECURITY failed on init: %ld.  Disabling FMP support.\n",
+			 res.a0);
+		return;
+	}
+	ufshcd_set_sg_entry_size(hba, sizeof(struct fmp_sg_entry));
+
+	/*
+	 * This is needed to initialize FMP.  Without it, errors occur when
+	 * inline encryption is used.
+	 */
+	arm_smccc_smc(SMC_CMD_SMU, SMU_INIT, SMU_EMBEDDED, 0, 0, 0, 0, 0, &res);
+	if (res.a0) {
+		dev_err(hba->dev,
+			"SMC_CMD_SMU(SMU_INIT) failed: %ld.  Disabling FMP support.\n",
+			res.a0);
+		return;
+	}
+
+	/* Advertise crypto capabilities to the block layer. */
+	err = devm_blk_crypto_profile_init(hba->dev, profile, 0);
+	if (err) {
+		/* Only ENOMEM should be possible here. */
+		dev_err(hba->dev, "Failed to initialize crypto profile: %d\n",
+			err);
+		return;
+	}
+	profile->max_dun_bytes_supported = AES_BLOCK_SIZE;
+	profile->dev = hba->dev;
+	profile->modes_supported[BLK_ENCRYPTION_MODE_AES_256_XTS] =
+		DATA_UNIT_SIZE;
+
+	/* Advertise crypto support to ufshcd-core. */
+	hba->caps |= UFSHCD_CAP_CRYPTO;
+
+	/* Advertise crypto quirks to ufshcd-core. */
+	hba->quirks |= UFSHCD_QUIRK_CUSTOM_CRYPTO_PROFILE |
+		       UFSHCD_QUIRK_BROKEN_CRYPTO_ENABLE |
+		       UFSHCD_QUIRK_KEYS_IN_PRDT;
+
+}
+
+static void exynos_ufs_fmp_resume(struct ufs_hba *hba)
+{
+	struct arm_smccc_res res;
+
+	arm_smccc_smc(SMC_CMD_FMP_SECURITY, 0, SMU_EMBEDDED, CFG_DESCTYPE_3,
+		      0, 0, 0, 0, &res);
+	if (res.a0)
+		dev_err(hba->dev,
+			"SMC_CMD_FMP_SECURITY failed on resume: %ld\n", res.a0);
+
+	arm_smccc_smc(SMC_CMD_FMP_SMU_RESUME, 0, SMU_EMBEDDED, 0, 0, 0, 0, 0,
+		      &res);
+	if (res.a0)
+		dev_err(hba->dev,
+			"SMC_CMD_FMP_SMU_RESUME failed: %ld\n", res.a0);
+}
+
+static inline __be64 fmp_key_word(const u8 *key, int j)
+{
+	return cpu_to_be64(get_unaligned_le64(
+			key + AES_KEYSIZE_256 - (j + 1) * sizeof(u64)));
+}
+
+/* Fill the PRDT for a request according to the given encryption context. */
+static int exynos_ufs_fmp_fill_prdt(struct ufs_hba *hba,
+				    const struct bio_crypt_ctx *crypt_ctx,
+				    void *prdt, unsigned int num_segments)
+{
+	struct fmp_sg_entry *fmp_prdt = prdt;
+	const u8 *enckey = crypt_ctx->bc_key->raw;
+	const u8 *twkey = enckey + AES_KEYSIZE_256;
+	u64 dun_lo = crypt_ctx->bc_dun[0];
+	u64 dun_hi = crypt_ctx->bc_dun[1];
+	unsigned int i;
+
+	/* If FMP wasn't enabled, we shouldn't get any encrypted requests. */
+	if (WARN_ON_ONCE(!(hba->caps & UFSHCD_CAP_CRYPTO)))
+		return -EIO;
+
+	/* Configure FMP on each segment of the request. */
+	for (i = 0; i < num_segments; i++) {
+		struct fmp_sg_entry *prd = &fmp_prdt[i];
+		int j;
+
+		/* Each segment must be exactly one data unit. */
+		if (prd->base.size != cpu_to_le32(DATA_UNIT_SIZE - 1)) {
+			dev_err(hba->dev,
+				"data segment is misaligned for FMP\n");
+			return -EIO;
+		}
+
+		/* Set the algorithm and key length. */
+		prd->base.size |= cpu_to_le32((FMP_ALGO_MODE_AES_XTS << 28) |
+					      (FMP_KEYLEN_256BIT << 26));
+
+		/* Set the IV. */
+		prd->file_iv[0] = cpu_to_be64(dun_hi);
+		prd->file_iv[1] = cpu_to_be64(dun_lo);
+
+		/* Set the key. */
+		for (j = 0; j < AES_KEYSIZE_256 / sizeof(u64); j++) {
+			prd->file_enckey[j] = fmp_key_word(enckey, j);
+			prd->file_twkey[j] = fmp_key_word(twkey, j);
+		}
+
+		/* Increment the data unit number. */
+		dun_lo++;
+		if (dun_lo == 0)
+			dun_hi++;
+	}
+	return 0;
+}
+
+#else /* CONFIG_SCSI_UFS_CRYPTO */
+
+static void exynos_ufs_fmp_init(struct ufs_hba *hba)
+{
+}
+
+static void exynos_ufs_fmp_resume(struct ufs_hba *hba)
+{
+}
+
+#define exynos_ufs_fmp_fill_prdt NULL
+
+#endif /* !CONFIG_SCSI_UFS_CRYPTO */
+
 static int exynos_ufs_init(struct ufs_hba *hba)
 {
 	struct device *dev = hba->dev;
 	struct platform_device *pdev = to_platform_device(dev);
 	struct exynos_ufs *ufs;
@@ -1196,10 +1409,12 @@ static int exynos_ufs_init(struct ufs_hba *hba)
 		goto out;
 	}
 
 	exynos_ufs_priv_init(hba, ufs);
 
+	exynos_ufs_fmp_init(hba);
+
 	if (ufs->drv_data->drv_init) {
 		ret = ufs->drv_data->drv_init(dev, ufs);
 		if (ret) {
 			dev_err(dev, "failed to init drv-data\n");
 			goto out;
@@ -1211,11 +1426,11 @@ static int exynos_ufs_init(struct ufs_hba *hba)
 		goto out;
 	exynos_ufs_specify_phy_time_attr(ufs);
 	if (!(ufs->opts & EXYNOS_UFS_OPT_UFSPR_SECURE))
 		exynos_ufs_config_smu(ufs);
 
-	hba->host->dma_alignment = SZ_4K - 1;
+	hba->host->dma_alignment = DATA_UNIT_SIZE - 1;
 	return 0;
 
 out:
 	hba->priv = NULL;
 	return ret;
@@ -1330,11 +1545,11 @@ static int exynos_ufs_hce_enable_notify(struct ufs_hba *hba,
 		 * The maximum segment size must be set after scsi_host_alloc()
 		 * has been called and before LUN scanning starts
 		 * (ufshcd_async_scan()). Note: this callback may also be called
 		 * from other functions than ufshcd_init().
 		 */
-		hba->host->max_segment_size = SZ_4K;
+		hba->host->max_segment_size = DATA_UNIT_SIZE;
 
 		if (ufs->drv_data->pre_hce_enable) {
 			ret = ufs->drv_data->pre_hce_enable(ufs);
 			if (ret)
 				return ret;
@@ -1430,11 +1645,11 @@ static int exynos_ufs_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 
 	if (!ufshcd_is_link_active(hba))
 		phy_power_on(ufs->phy);
 
 	exynos_ufs_config_smu(ufs);
-
+	exynos_ufs_fmp_resume(hba);
 	return 0;
 }
 
 static int exynosauto_ufs_vh_link_startup_notify(struct ufs_hba *hba,
 						 enum ufs_notify_change_status status)
@@ -1696,10 +1911,11 @@ static const struct ufs_hba_variant_ops ufs_hba_exynos_ops = {
 	.setup_xfer_req			= exynos_ufs_specify_nexus_t_xfer_req,
 	.setup_task_mgmt		= exynos_ufs_specify_nexus_t_tm_req,
 	.hibern8_notify			= exynos_ufs_hibern8_notify,
 	.suspend			= exynos_ufs_suspend,
 	.resume				= exynos_ufs_resume,
+	.fill_crypto_prdt		= exynos_ufs_fmp_fill_prdt,
 };
 
 static struct ufs_hba_variant_ops ufs_hba_exynosauto_vh_ops = {
 	.name				= "exynosauto_ufs_vh",
 	.init				= exynosauto_ufs_vh_init,
-- 
2.45.2

Re: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

[Bart Van Assche]

On 7/2/24 12:25 AM, Eric Biggers wrote:
> From: Eric Biggers <ebiggers@google.com>
> 
> Add support for Flash Memory Protector (FMP), which is the inline
> encryption hardware on Exynos and Exynos-based SoCs.
> 
> Specifically, add support for the "traditional FMP mode" that works on
> many Exynos-based SoCs including gs101.  This is the mode that uses
> "software keys" and is compatible with the upstream kernel's existing
> inline encryption framework in the block and filesystem layers.  I plan
> to add support for the wrapped key support on gs101 at a later time.
> 
> Tested on gs101 (specifically Pixel 6) by running the 'encrypt' group of
> xfstests on a filesystem mounted with the 'inlinecrypt' mount option.
> 
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---
>   drivers/ufs/host/ufs-exynos.c | 228 +++++++++++++++++++++++++++++++++-
>   1 file changed, 222 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/ufs/host/ufs-exynos.c b/drivers/ufs/host/ufs-exynos.c
> index 88d125d1ee3c..dd545ef7c361 100644
> --- a/drivers/ufs/host/ufs-exynos.c
> +++ b/drivers/ufs/host/ufs-exynos.c
> @@ -6,10 +6,13 @@
>    * Author: Seungwon Jeon  <essuuj@gmail.com>
>    * Author: Alim Akhtar <alim.akhtar@samsung.com>
>    *
>    */
>   
> +#include <asm/unaligned.h>
> +#include <crypto/aes.h>
> +#include <linux/arm-smccc.h>
>   #include <linux/clk.h>
>   #include <linux/delay.h>
>   #include <linux/module.h>
>   #include <linux/of.h>
>   #include <linux/of_address.h>
> @@ -23,16 +26,18 @@
>   #include <ufs/ufshci.h>
>   #include <ufs/unipro.h>
>   
>   #include "ufs-exynos.h"
>   
> +#define DATA_UNIT_SIZE		4096
> +#define LOG2_DATA_UNIT_SIZE	12

If this series has to be reposted, please consider changing "12" into
"ilog2(DATA_UNIT_SIZE)". I think that the ilog2() macro generates a
constant expression if its argument is a constant.

In case it wouldn't be clear, I'm fine with this patch with or without
that change.

Thanks,

Bart.

Re: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

[Peter Griffin]

Hi Eric,

Thanks for your contribution, it's great to see new features like this
being posted upstream for gs101 :)

On Tue, 2 Jul 2024 at 08:28, Eric Biggers <ebiggers@kernel.org> wrote:
>
> From: Eric Biggers <ebiggers@google.com>
>
> Add support for Flash Memory Protector (FMP), which is the inline
> encryption hardware on Exynos and Exynos-based SoCs.
>
> Specifically, add support for the "traditional FMP mode" that works on
> many Exynos-based SoCs including gs101.  This is the mode that uses
> "software keys" and is compatible with the upstream kernel's existing
> inline encryption framework in the block and filesystem layers.  I plan
> to add support for the wrapped key support on gs101 at a later time.
>
> Tested on gs101 (specifically Pixel 6) by running the 'encrypt' group of
> xfstests on a filesystem mounted with the 'inlinecrypt' mount option.
>
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---
>  drivers/ufs/host/ufs-exynos.c | 228 +++++++++++++++++++++++++++++++++-
>  1 file changed, 222 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/ufs/host/ufs-exynos.c b/drivers/ufs/host/ufs-exynos.c
> index 88d125d1ee3c..dd545ef7c361 100644
> --- a/drivers/ufs/host/ufs-exynos.c
> +++ b/drivers/ufs/host/ufs-exynos.c
> @@ -6,10 +6,13 @@
>   * Author: Seungwon Jeon  <essuuj@gmail.com>
>   * Author: Alim Akhtar <alim.akhtar@samsung.com>
>   *
>   */
>
> +#include <asm/unaligned.h>
> +#include <crypto/aes.h>
> +#include <linux/arm-smccc.h>
>  #include <linux/clk.h>
>  #include <linux/delay.h>
>  #include <linux/module.h>
>  #include <linux/of.h>
>  #include <linux/of_address.h>
> @@ -23,16 +26,18 @@
>  #include <ufs/ufshci.h>
>  #include <ufs/unipro.h>
>
>  #include "ufs-exynos.h"
>
> +#define DATA_UNIT_SIZE         4096
> +#define LOG2_DATA_UNIT_SIZE    12
> +
>  /*
>   * Exynos's Vendor specific registers for UFSHCI
>   */
>  #define HCI_TXPRDT_ENTRY_SIZE  0x00
>  #define PRDT_PREFECT_EN                BIT(31)
> -#define PRDT_SET_SIZE(x)       ((x) & 0x1F)
>  #define HCI_RXPRDT_ENTRY_SIZE  0x04
>  #define HCI_1US_TO_CNT_VAL     0x0C
>  #define CNT_VAL_1US_MASK       0x3FF
>  #define HCI_UTRL_NEXUS_TYPE    0x40
>  #define HCI_UTMRL_NEXUS_TYPE   0x44
> @@ -1041,12 +1046,12 @@ static int exynos_ufs_post_link(struct ufs_hba *hba)
>
>         exynos_ufs_establish_connt(ufs);
>         exynos_ufs_fit_aggr_timeout(ufs);
>
>         hci_writel(ufs, 0xa, HCI_DATA_REORDER);
> -       hci_writel(ufs, PRDT_SET_SIZE(12), HCI_TXPRDT_ENTRY_SIZE);
> -       hci_writel(ufs, PRDT_SET_SIZE(12), HCI_RXPRDT_ENTRY_SIZE);
> +       hci_writel(ufs, LOG2_DATA_UNIT_SIZE, HCI_TXPRDT_ENTRY_SIZE);
> +       hci_writel(ufs, LOG2_DATA_UNIT_SIZE, HCI_RXPRDT_ENTRY_SIZE);
>         hci_writel(ufs, (1 << hba->nutrs) - 1, HCI_UTRL_NEXUS_TYPE);
>         hci_writel(ufs, (1 << hba->nutmrs) - 1, HCI_UTMRL_NEXUS_TYPE);
>         hci_writel(ufs, 0xf, HCI_AXIDMA_RWDATA_BURST_LEN);
>
>         if (ufs->opts & EXYNOS_UFS_OPT_SKIP_CONNECTION_ESTAB)
> @@ -1149,10 +1154,218 @@ static inline void exynos_ufs_priv_init(struct ufs_hba *hba,
>                 ufs->rx_sel_idx = 0;
>         hba->priv = (void *)ufs;
>         hba->quirks = ufs->drv_data->quirks;
>  }
>
> +#ifdef CONFIG_SCSI_UFS_CRYPTO
> +
> +/*
> + * Support for Flash Memory Protector (FMP), which is the inline encryption
> + * hardware on Exynos and Exynos-based SoCs.  The interface to this hardware is
> + * not compatible with the standard UFS crypto.  It requires that encryption be
> + * configured in the PRDT using a nonstandard extension.
> + */
> +
> +enum fmp_crypto_algo_mode {
> +       FMP_BYPASS_MODE = 0,
> +       FMP_ALGO_MODE_AES_CBC = 1,
> +       FMP_ALGO_MODE_AES_XTS = 2,
> +};
> +enum fmp_crypto_key_length {
> +       FMP_KEYLEN_256BIT = 1,
> +};
> +
> +/**
> + * struct fmp_sg_entry - nonstandard format of PRDT entries when FMP is enabled
> + *
> + * @base: The standard PRDT entry, but with nonstandard bitfields in the high
> + *     bits of the 'size' field, i.e. the last 32-bit word.  When these
> + *     nonstandard bitfields are zero, the data segment won't be encrypted or
> + *     decrypted.  Otherwise they specify the algorithm and key length with
> + *     which the data segment will be encrypted or decrypted.
> + * @file_iv: The initialization vector (IV) with all bytes reversed
> + * @file_enckey: The first half of the AES-XTS key with all bytes reserved
> + * @file_twkey: The second half of the AES-XTS key with all bytes reserved
> + * @disk_iv: Unused
> + * @reserved: Unused
> + */
> +struct fmp_sg_entry {
> +       struct ufshcd_sg_entry base;
> +       __be64 file_iv[2];
> +       __be64 file_enckey[4];
> +       __be64 file_twkey[4];
> +       __be64 disk_iv[2];
> +       __be64 reserved[2];
> +};
> +
> +#define SMC_CMD_FMP_SECURITY   \
> +       ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
> +                          ARM_SMCCC_OWNER_SIP, 0x1810)
> +#define SMC_CMD_SMU            \
> +       ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
> +                          ARM_SMCCC_OWNER_SIP, 0x1850)
> +#define SMC_CMD_FMP_SMU_RESUME \
> +       ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \
> +                          ARM_SMCCC_OWNER_SIP, 0x1860)
> +#define SMU_EMBEDDED                   0
> +#define SMU_INIT                       0
> +#define CFG_DESCTYPE_3                 3
> +
> +static void exynos_ufs_fmp_init(struct ufs_hba *hba)
> +{
> +       struct blk_crypto_profile *profile = &hba->crypto_profile;
> +       struct arm_smccc_res res;
> +       int err;
> +
> +       /*
> +        * Check for the standard crypto support bit, since it's available even
> +        * though the rest of the interface to FMP is nonstandard.
> +        *
> +        * This check should have the effect of preventing the driver from
> +        * trying to use FMP on old Exynos SoCs that don't have FMP.
> +        */
> +       if (!(ufshcd_readl(hba, REG_CONTROLLER_CAPABILITIES) &
> +             MASK_CRYPTO_SUPPORT))
> +               return;
> +

Do you know how these FMP registers (FMPSECURITY0 etc) relate to the
UFSPR* registers set in the existing exynos_ufs_config_smu()? The
UFS_LINK spec talks about UFSPR(FMP), so I had assumed the FMP support
would be writing these same registers but via SMC call.

I think by the looks of things

#define UFSPRSECURITY 0x010
#define UFSPSBEGIN0 0x200
#define UFSPSEND0 0x204
#define UFSPSLUN0 0x208
#define UFSPSCTRL0 0x20C

relates to the following registers in gs101 spec

FMPSECURITY0 0x0010
FMPSBEGIN0 0x2000
FMPSEND0 0x2004
FMPSLUN0 0x2008
FMPSCTRL0 0x200C

And the SMC calls your calling set those same registers as
exynos_ufs_config_smu() function. Although it is hard to be certain as
I don't have access to the firmware code. Certainly the comment below
about FMPSECURITY0 implies that :)

With that in mind I think exynos_ufs_fmp_init() function in this patch
needs to be better integrated with the EXYNOS_UFS_OPT_UFSPR_SECURE
flag and the existing exynos_ufs_config_smu() function that is
currently just disabling decryption on platforms where it can access
the UFSPR(FMP) regs via mmio.

Thanks,

Peter

p.s. Also whilst reviewing this I noticed a bug where I don't check
EXYNOS_UFS_OPT_UFSPR_SECURE flag in exynos_ufs_resume() before calling
exynos_ufs_config_smu(). I'll send a patch to fix that.

[..]

Re: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

[Eric Biggers]

Hi Peter,

On Thu, Jul 04, 2024 at 02:26:05PM +0100, Peter Griffin wrote:
> Do you know how these FMP registers (FMPSECURITY0 etc) relate to the
> UFSPR* registers set in the existing exynos_ufs_config_smu()? The
> UFS_LINK spec talks about UFSPR(FMP), so I had assumed the FMP support
> would be writing these same registers but via SMC call.
> 
> I think by the looks of things
> 
> #define UFSPRSECURITY 0x010
> #define UFSPSBEGIN0 0x200
> #define UFSPSEND0 0x204
> #define UFSPSLUN0 0x208
> #define UFSPSCTRL0 0x20C
> 
> relates to the following registers in gs101 spec
> 
> FMPSECURITY0 0x0010
> FMPSBEGIN0 0x2000
> FMPSEND0 0x2004
> FMPSLUN0 0x2008
> FMPSCTRL0 0x200C
> 
> And the SMC calls your calling set those same registers as
> exynos_ufs_config_smu() function. Although it is hard to be certain as
> I don't have access to the firmware code. Certainly the comment below
> about FMPSECURITY0 implies that :)
> 
> With that in mind I think exynos_ufs_fmp_init() function in this patch
> needs to be better integrated with the EXYNOS_UFS_OPT_UFSPR_SECURE
> flag and the existing exynos_ufs_config_smu() function that is
> currently just disabling decryption on platforms where it can access
> the UFSPR(FMP) regs via mmio.

I think that is all correct.  For some reason, on gs101 the FMP registers are
not accessible by the "normal world", and SMC calls need to be used instead.
The sequences of SMC calls originated from Samsung's Linux driver code for FMP.
So I know they are the magic incantations that are needed, but I don't have
access to the source code or documentation for them.  It does seem clear that
one of the things they must do is write the needed values to the FMP registers.

I'd hope that these same SMC calls also work on Exynos-based SoCs that do make
the FMP registers accessible to the "normal world", and therefore they can just
be used on all Exynos-based SoCs and ufs-exynos won't need two different code
paths.  But I don't have a way to confirm this myself.  Until someone is able to
confirm this, I think we need to make the FMP support depend on
EXYNOS_UFS_OPT_UFSPR_SECURE so that it doesn't conflict with
exynos_ufs_config_smu() which runs when !EXYNOS_UFS_OPT_UFSPR_SECURE.

- Eric

Re: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

[Eric Biggers]

On Mon, Jul 08, 2024 at 01:26:30PM -0700, Eric Biggers wrote:
> Hi Peter,
> 
> On Thu, Jul 04, 2024 at 02:26:05PM +0100, Peter Griffin wrote:
> > Do you know how these FMP registers (FMPSECURITY0 etc) relate to the
> > UFSPR* registers set in the existing exynos_ufs_config_smu()? The
> > UFS_LINK spec talks about UFSPR(FMP), so I had assumed the FMP support
> > would be writing these same registers but via SMC call.
> > 
> > I think by the looks of things
> > 
> > #define UFSPRSECURITY 0x010
> > #define UFSPSBEGIN0 0x200
> > #define UFSPSEND0 0x204
> > #define UFSPSLUN0 0x208
> > #define UFSPSCTRL0 0x20C
> > 
> > relates to the following registers in gs101 spec
> > 
> > FMPSECURITY0 0x0010
> > FMPSBEGIN0 0x2000
> > FMPSEND0 0x2004
> > FMPSLUN0 0x2008
> > FMPSCTRL0 0x200C
> > 
> > And the SMC calls your calling set those same registers as
> > exynos_ufs_config_smu() function. Although it is hard to be certain as
> > I don't have access to the firmware code. Certainly the comment below
> > about FMPSECURITY0 implies that :)
> > 
> > With that in mind I think exynos_ufs_fmp_init() function in this patch
> > needs to be better integrated with the EXYNOS_UFS_OPT_UFSPR_SECURE
> > flag and the existing exynos_ufs_config_smu() function that is
> > currently just disabling decryption on platforms where it can access
> > the UFSPR(FMP) regs via mmio.
> 
> I think that is all correct.  For some reason, on gs101 the FMP registers are
> not accessible by the "normal world", and SMC calls need to be used instead.
> The sequences of SMC calls originated from Samsung's Linux driver code for FMP.
> So I know they are the magic incantations that are needed, but I don't have
> access to the source code or documentation for them.  It does seem clear that
> one of the things they must do is write the needed values to the FMP registers.
> 
> I'd hope that these same SMC calls also work on Exynos-based SoCs that do make
> the FMP registers accessible to the "normal world", and therefore they can just
> be used on all Exynos-based SoCs and ufs-exynos won't need two different code
> paths.  But I don't have a way to confirm this myself.  Until someone is able to
> confirm this, I think we need to make the FMP support depend on
> EXYNOS_UFS_OPT_UFSPR_SECURE so that it doesn't conflict with
> exynos_ufs_config_smu() which runs when !EXYNOS_UFS_OPT_UFSPR_SECURE.
> 

These same SMC calls can be found in the downstream source for other
Exynos-based SoCs.  I suspect that exynos_ufs_config_smu() should be removed,
and exynos_ufs_fmp_init() should run regardless of EXYNOS_UFS_OPT_UFSPR_SECURE.
It still would need to be tested, though, which I'm not able to do.  (And
especially as a cryptography feature, this *must* be tested...)  So for now I'm
going to make the FMP support conditional on EXYNOS_UFS_OPT_UFSPR_SECURE.

- Eric

RE: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

[Alim Akhtar]

Hello Eric,

> -----Original Message-----
> From: Eric Biggers <ebiggers@kernel.org>
> Sent: Tuesday, July 9, 2024 5:19 AM
> To: Peter Griffin <peter.griffin@linaro.org>
> Cc: linux-scsi@vger.kernel.org; linux-samsung-soc@vger.kernel.org; linux-
> fscrypt@vger.kernel.org; Alim Akhtar <alim.akhtar@samsung.com>; Avri
> Altman <avri.altman@wdc.com>; Bart Van Assche <bvanassche@acm.org>;
> Martin K . Petersen <martin.petersen@oracle.com>; André Draszik
> <andre.draszik@linaro.org>; William McVicker <willmcvicker@google.com>
> Subject: Re: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash
Memory
> Protector (FMP)
> 
> On Mon, Jul 08, 2024 at 01:26:30PM -0700, Eric Biggers wrote:
> > Hi Peter,
> >
> > On Thu, Jul 04, 2024 at 02:26:05PM +0100, Peter Griffin wrote:
> > > Do you know how these FMP registers (FMPSECURITY0 etc) relate to the
> > > UFSPR* registers set in the existing exynos_ufs_config_smu()? The
> > > UFS_LINK spec talks about UFSPR(FMP), so I had assumed the FMP
> > > support would be writing these same registers but via SMC call.
> > >
> > > I think by the looks of things
> > >
> > > #define UFSPRSECURITY 0x010
> > > #define UFSPSBEGIN0 0x200
> > > #define UFSPSEND0 0x204
> > > #define UFSPSLUN0 0x208
> > > #define UFSPSCTRL0 0x20C
> > >
> > > relates to the following registers in gs101 spec
> > >
> > > FMPSECURITY0 0x0010
> > > FMPSBEGIN0 0x2000
> > > FMPSEND0 0x2004
> > > FMPSLUN0 0x2008
> > > FMPSCTRL0 0x200C
> > >
> > > And the SMC calls your calling set those same registers as
> > > exynos_ufs_config_smu() function. Although it is hard to be certain
> > > as I don't have access to the firmware code. Certainly the comment
> > > below about FMPSECURITY0 implies that :)
> > >
> > > With that in mind I think exynos_ufs_fmp_init() function in this
> > > patch needs to be better integrated with the
> > > EXYNOS_UFS_OPT_UFSPR_SECURE flag and the existing
> > > exynos_ufs_config_smu() function that is currently just disabling
> > > decryption on platforms where it can access the UFSPR(FMP) regs via
> mmio.
> >
> > I think that is all correct.  For some reason, on gs101 the FMP
> > registers are not accessible by the "normal world", and SMC calls need
to
> be used instead.
> > The sequences of SMC calls originated from Samsung's Linux driver code
> for FMP.
> > So I know they are the magic incantations that are needed, but I don't
> > have access to the source code or documentation for them.  It does
> > seem clear that one of the things they must do is write the needed
values
> to the FMP registers.
> >
> > I'd hope that these same SMC calls also work on Exynos-based SoCs that
> > do make the FMP registers accessible to the "normal world", and
> > therefore they can just be used on all Exynos-based SoCs and
> > ufs-exynos won't need two different code paths.  But I don't have a
> > way to confirm this myself.  Until someone is able to confirm this, I
> > think we need to make the FMP support depend on
> > EXYNOS_UFS_OPT_UFSPR_SECURE so that it doesn't conflict with
> > exynos_ufs_config_smu() which runs when
> !EXYNOS_UFS_OPT_UFSPR_SECURE.
> >
> 
> These same SMC calls can be found in the downstream source for other
> Exynos-based SoCs.  I suspect that exynos_ufs_config_smu() should be
> removed, and exynos_ufs_fmp_init() should run regardless of
> EXYNOS_UFS_OPT_UFSPR_SECURE.
> It still would need to be tested, though, which I'm not able to do.  (And
> especially as a cryptography feature, this *must* be tested...)  So for
now I'm
> going to make the FMP support conditional on
> EXYNOS_UFS_OPT_UFSPR_SECURE.
> 
SMU controls the security access aspect of the FMP, one can have a usecase
where one wants to enable inline encryption using FMP in a non-secure
mode/world after a secure boot of the system
and in another case, configure FMP in secure mode/world during secure boot.
I am not sure how it is designed in gs101 though.
Currently, exynos_ufs_config_smu() just allows SMU registers modification by
non-secure world.

> - Eric

Re: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash Memory Protector (FMP)

[Eric Biggers]

On Wed, Jul 10, 2024 at 11:22:52AM +0530, Alim Akhtar wrote:
> Hello Eric,
> 
> > -----Original Message-----
> > From: Eric Biggers <ebiggers@kernel.org>
> > Sent: Tuesday, July 9, 2024 5:19 AM
> > To: Peter Griffin <peter.griffin@linaro.org>
> > Cc: linux-scsi@vger.kernel.org; linux-samsung-soc@vger.kernel.org; linux-
> > fscrypt@vger.kernel.org; Alim Akhtar <alim.akhtar@samsung.com>; Avri
> > Altman <avri.altman@wdc.com>; Bart Van Assche <bvanassche@acm.org>;
> > Martin K . Petersen <martin.petersen@oracle.com>; André Draszik
> > <andre.draszik@linaro.org>; William McVicker <willmcvicker@google.com>
> > Subject: Re: [PATCH v2 6/6] scsi: ufs: exynos: Add support for Flash
> Memory
> > Protector (FMP)
> > 
> > On Mon, Jul 08, 2024 at 01:26:30PM -0700, Eric Biggers wrote:
> > > Hi Peter,
> > >
> > > On Thu, Jul 04, 2024 at 02:26:05PM +0100, Peter Griffin wrote:
> > > > Do you know how these FMP registers (FMPSECURITY0 etc) relate to the
> > > > UFSPR* registers set in the existing exynos_ufs_config_smu()? The
> > > > UFS_LINK spec talks about UFSPR(FMP), so I had assumed the FMP
> > > > support would be writing these same registers but via SMC call.
> > > >
> > > > I think by the looks of things
> > > >
> > > > #define UFSPRSECURITY 0x010
> > > > #define UFSPSBEGIN0 0x200
> > > > #define UFSPSEND0 0x204
> > > > #define UFSPSLUN0 0x208
> > > > #define UFSPSCTRL0 0x20C
> > > >
> > > > relates to the following registers in gs101 spec
> > > >
> > > > FMPSECURITY0 0x0010
> > > > FMPSBEGIN0 0x2000
> > > > FMPSEND0 0x2004
> > > > FMPSLUN0 0x2008
> > > > FMPSCTRL0 0x200C
> > > >
> > > > And the SMC calls your calling set those same registers as
> > > > exynos_ufs_config_smu() function. Although it is hard to be certain
> > > > as I don't have access to the firmware code. Certainly the comment
> > > > below about FMPSECURITY0 implies that :)
> > > >
> > > > With that in mind I think exynos_ufs_fmp_init() function in this
> > > > patch needs to be better integrated with the
> > > > EXYNOS_UFS_OPT_UFSPR_SECURE flag and the existing
> > > > exynos_ufs_config_smu() function that is currently just disabling
> > > > decryption on platforms where it can access the UFSPR(FMP) regs via
> > mmio.
> > >
> > > I think that is all correct.  For some reason, on gs101 the FMP
> > > registers are not accessible by the "normal world", and SMC calls need
> to
> > be used instead.
> > > The sequences of SMC calls originated from Samsung's Linux driver code
> > for FMP.
> > > So I know they are the magic incantations that are needed, but I don't
> > > have access to the source code or documentation for them.  It does
> > > seem clear that one of the things they must do is write the needed
> values
> > to the FMP registers.
> > >
> > > I'd hope that these same SMC calls also work on Exynos-based SoCs that
> > > do make the FMP registers accessible to the "normal world", and
> > > therefore they can just be used on all Exynos-based SoCs and
> > > ufs-exynos won't need two different code paths.  But I don't have a
> > > way to confirm this myself.  Until someone is able to confirm this, I
> > > think we need to make the FMP support depend on
> > > EXYNOS_UFS_OPT_UFSPR_SECURE so that it doesn't conflict with
> > > exynos_ufs_config_smu() which runs when
> > !EXYNOS_UFS_OPT_UFSPR_SECURE.
> > >
> > 
> > These same SMC calls can be found in the downstream source for other
> > Exynos-based SoCs.  I suspect that exynos_ufs_config_smu() should be
> > removed, and exynos_ufs_fmp_init() should run regardless of
> > EXYNOS_UFS_OPT_UFSPR_SECURE.
> > It still would need to be tested, though, which I'm not able to do.  (And
> > especially as a cryptography feature, this *must* be tested...)  So for
> now I'm
> > going to make the FMP support conditional on
> > EXYNOS_UFS_OPT_UFSPR_SECURE.
> > 
> SMU controls the security access aspect of the FMP, one can have a usecase
> where one wants to enable inline encryption using FMP in a non-secure
> mode/world after a secure boot of the system
> and in another case, configure FMP in secure mode/world during secure boot.
> I am not sure how it is designed in gs101 though.
> Currently, exynos_ufs_config_smu() just allows SMU registers modification by
> non-secure world.
> 

Apparently, gs101 has two levels of access control for FMP.  Access to the range
configuration registers like FMPSBEGIN0 (UFSPSBEGIN0 in the upstream source) is
controlled by FMPSECURITY0.NSSMU (UFSPRSECURITY.NSSMU in the upstream source).
But access to FMPSECURITY0 itself is controlled by TZPC.NSFMP, which is writable
only by the "secure world".  In the current upstream source,
exynos_ufs_config_smu() writes to FMPSECURITY0, and this crashes on gs101 if
actually executed (it's currently disabled on gs101 for that reason).  So the
secure world software on gs101 must be setting TZPC.NSFMP = 0.

I expect that this isn't specific to gs101, and some other Exynos-based SoCs use
this same design.  This would explain the presence of the SMC calls in the
downstream source used on other Exynos-based SoCs.

So it seems that for now ufs-exynos has to use the SMC calls, as this patch
does.  This is similar to how ufs-qcom and ufs-mediatek similarly use their
SoC's respective set of SMC calls to configure inline encryption, in order to
work around similar designs where the inline encryption hardware can only be
configured by secure world software.  (I don't know why so many SoC vendors are
choosing this design, given that in practice only the normal world wants to
configure inline encryption.  The detour to the secure world seems unnecessary.)

- Eric

Re: [PATCH v2 0/6] Basic inline encryption support for ufs-exynos

[Bart Van Assche]

On 7/2/24 12:25 AM, Eric Biggers wrote:
> Add support for Flash Memory Protector (FMP), which is the inline
> encryption hardware on Exynos and Exynos-based SoCs.
> 
> Specifically, add support for the "traditional FMP mode" that works on
> many Exynos-based SoCs including gs101.  This is the mode that uses
> "software keys" and is compatible with the upstream kernel's existing
> inline encryption framework in the block and filesystem layers.  I plan
> to add support for the wrapped key support on gs101 at a later time.
> 
> Tested on gs101 (specifically Pixel 6) by running the 'encrypt' group of
> xfstests on a filesystem mounted with the 'inlinecrypt' mount option.
> 
> This patchset applies to v6.10-rc6, and it has no prerequisites that
> aren't already upstream.

For the entire series:

Reviewed-by: Bart Van Assche <bvanassche@acm.org>