[PATCH v2 0/4] xfstests: test custom crypto data unit size

[PATCH v2 0/4] xfstests: test custom crypto data unit size

[Eric Biggers]

This series adds a test that verifies the on-disk format of encrypted
files that use a crypto data unit size that differs from the filesystem
block size.  This tests the functionality that was introduced in Linux
6.7 by kernel commit 5b1188847180 ("fscrypt: support crypto data unit
size less than filesystem block size").

This depends on the xfsprogs patch
"xfs_io/encrypt: support specifying crypto data unit size"
(https://lore.kernel.org/r/20231013062639.141468-1-ebiggers@kernel.org)
which adds the '-s' option to the set_encpolicy command of xfs_io.

As usual, the test skips itself when any prerequisite isn't met.

I've tested the new test on both ext4 and f2fs.

Changed in v2:
- Updated the cover letter, commit message, and a comment to reflect
  that the kernel commit that added this feature was merged in 6.7.
- Rebased onto latest for-next branch of xfstests.

Eric Biggers (4):
  fscrypt-crypt-util: rename block to data unit
  common/rc: fix _require_xfs_io_command with digits in argument
  common/encrypt: support custom data unit size
  generic: add test for custom crypto data unit size

 common/encrypt           | 42 +++++++++++++-----
 common/rc                |  2 +-
 src/fscrypt-crypt-util.c | 93 ++++++++++++++++++++--------------------
 tests/f2fs/002           |  6 +--
 tests/generic/900        | 29 +++++++++++++
 tests/generic/900.out    | 11 +++++
 6 files changed, 123 insertions(+), 60 deletions(-)
 create mode 100755 tests/generic/900
 create mode 100644 tests/generic/900.out


base-commit: b9e1a88f8198ac02f3b82fe3b127d4e14f4a97b7
-- 
2.42.1

[PATCH v2 1/4] fscrypt-crypt-util: rename block to data unit

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Rename the --block-size option to --data-unit-size, and rename the
--block-number option to --data-unit-index.

This does not change any functionality, but this avoids confusion now
that the kernel supports the case where the crypto data unit size is not
the same as the filesystem block size.  fscrypt-crypt-util cares about
the crypto data unit size, not the filesystem block size.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 common/encrypt           | 10 ++---
 src/fscrypt-crypt-util.c | 93 ++++++++++++++++++++--------------------
 tests/f2fs/002           |  6 +--
 3 files changed, 55 insertions(+), 54 deletions(-)

diff --git a/common/encrypt b/common/encrypt
index 1a77e23b..5688745c 100644
--- a/common/encrypt
+++ b/common/encrypt
@@ -756,51 +756,51 @@ _do_verify_ciphertext_for_encryption_policy()
 
 	# Now unmount the filesystem and verify the ciphertext we just wrote.
 	_scratch_unmount
 
 	echo "Verifying encrypted file contents" >> $seqres.full
 	for f in "${test_contents_files[@]}"; do
 		read -r src inode blocklist <<< "$f"
 		nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode)
 		_dump_ciphertext_blocks $SCRATCH_DEV $blocklist > $tmp.actual_contents
 		$crypt_contents_cmd $contents_encryption_mode $raw_key_hex \
-			--file-nonce=$nonce --block-size=$blocksize \
+			--file-nonce=$nonce --data-unit-size=$blocksize \
 			--inode-number=$inode < $src > $tmp.expected_contents
 		if ! cmp $tmp.expected_contents $tmp.actual_contents; then
 			_fail "Expected encrypted contents != actual encrypted contents.  File: $f"
 		fi
 		$crypt_contents_cmd $contents_encryption_mode $raw_key_hex \
-			--decrypt --file-nonce=$nonce --block-size=$blocksize \
-			--inode-number=$inode \
+			--decrypt --file-nonce=$nonce \
+                        --data-unit-size=$blocksize --inode-number=$inode \
 			< $tmp.actual_contents > $tmp.decrypted_contents
 		if ! cmp $src $tmp.decrypted_contents; then
 			_fail "Contents decryption sanity check failed.  File: $f"
 		fi
 	done
 
 	echo "Verifying encrypted file names" >> $seqres.full
 	for f in "${test_filenames_files[@]}"; do
 		read -r name inode dir_inode padding <<< "$f"
 		nonce=$(_get_encryption_nonce $SCRATCH_DEV $dir_inode)
 		_get_ciphertext_filename $SCRATCH_DEV $inode $dir_inode \
 			> $tmp.actual_name
 		echo -n "$name" | \
 			$crypt_filename_cmd $filenames_encryption_mode \
 			$raw_key_hex --file-nonce=$nonce --padding=$padding \
-			--block-size=255 --inode-number=$dir_inode \
+			--data-unit-size=255 --inode-number=$dir_inode \
 			> $tmp.expected_name
 		if ! cmp $tmp.expected_name $tmp.actual_name; then
 			_fail "Expected encrypted filename != actual encrypted filename.  File: $f"
 		fi
 		$crypt_filename_cmd $filenames_encryption_mode $raw_key_hex \
 			--decrypt --file-nonce=$nonce --padding=$padding \
-			--block-size=255 --inode-number=$dir_inode \
+			--data-unit-size=255 --inode-number=$dir_inode \
 			< $tmp.actual_name > $tmp.decrypted_name
 		decrypted_name=$(tr -d '\0' < $tmp.decrypted_name)
 		if [ "$name" != "$decrypted_name" ]; then
 			_fail "Filename decryption sanity check failed ($name != $decrypted_name).  File: $f"
 		fi
 	done
 }
 
 # fscrypt UAPI constants (see <linux/fscrypt.h>)
 
diff --git a/src/fscrypt-crypt-util.c b/src/fscrypt-crypt-util.c
index 96f04799..a1b5005d 100644
--- a/src/fscrypt-crypt-util.c
+++ b/src/fscrypt-crypt-util.c
@@ -54,23 +54,23 @@ static void usage(FILE *fp)
 "MASTER_KEY (or a key derived from it, if a KDF is specified), and writes the\n"
 "resulting ciphertext (or plaintext) to stdout.\n"
 "\n"
 "CIPHER can be AES-256-XTS, AES-256-CTS-CBC, AES-128-CBC-ESSIV, AES-128-CTS-CBC,\n"
 "Adiantum, or AES-256-HCTR2.  MASTER_KEY must be a hex string long enough for\n"
 "the cipher.\n"
 "\n"
 "WARNING: this program is only meant for testing, not for \"real\" use!\n"
 "\n"
 "Options:\n"
-"  --block-number=BNUM         Starting block number for IV generation.\n"
+"  --data-unit-index=DUIDX     Starting data unit index for IV generation.\n"
 "                                Default: 0\n"
-"  --block-size=BLOCK_SIZE     Encrypt each BLOCK_SIZE bytes independently.\n"
+"  --data-unit-size=DUSIZE     Encrypt each DUSIZE bytes independently.\n"
 "                                Default: 4096 bytes\n"
 "  --decrypt                   Decrypt instead of encrypt\n"
 "  --direct-key                Use the format where the IVs include the file\n"
 "                                nonce and the same key is shared across files.\n"
 "  --dump-key-identifier       Instead of encrypting/decrypting data, just\n"
 "                                compute and dump the key identifier.\n"
 "  --file-nonce=NONCE          File's nonce as a 32-character hex string\n"
 "  --fs-uuid=UUID              The filesystem UUID as a 32-character hex string.\n"
 "                                Required for --iv-ino-lblk-32 and\n"
 "                                --iv-ino-lblk-64; otherwise is unused.\n"
@@ -79,22 +79,22 @@ static void usage(FILE *fp)
 "                                --iv-ino-lblk-32 and --iv-ino-lblk-64;\n"
 "                                otherwise is unused.\n"
 "  --iv-ino-lblk-32            Similar to --iv-ino-lblk-64, but selects the\n"
 "                                32-bit variant.\n"
 "  --iv-ino-lblk-64            Use the format where the IVs include the inode\n"
 "                                number and the same key is shared across files.\n"
 "  --kdf=KDF                   Key derivation function to use: AES-128-ECB,\n"
 "                                HKDF-SHA512, or none.  Default: none\n"
 "  --mode-num=NUM              The encryption mode number.  This may be required\n"
 "                                for key derivation, depending on other options.\n"
-"  --padding=PADDING           If last block is partial, zero-pad it to next\n"
-"                                PADDING-byte boundary.  Default: BLOCK_SIZE\n"
+"  --padding=PADDING           If last data unit is partial, zero-pad it to next\n"
+"                                PADDING-byte boundary.  Default: DUSIZE\n"
 	, fp);
 }
 
 /*----------------------------------------------------------------------------*
  *                                 Utilities                                  *
  *----------------------------------------------------------------------------*/
 
 #define ARRAY_SIZE(A)		(sizeof(A) / sizeof((A)[0]))
 #define MIN(x, y)		((x) < (y) ? (x) : (y))
 #define MAX(x, y)		((x) > (y) ? (x) : (y))
@@ -1968,73 +1968,74 @@ static const struct fscrypt_cipher *find_fscrypt_cipher(const char *name)
 	for (i = 0; i < ARRAY_SIZE(fscrypt_ciphers); i++) {
 		if (strcmp(fscrypt_ciphers[i].name, name) == 0)
 			return &fscrypt_ciphers[i];
 	}
 	return NULL;
 }
 
 union fscrypt_iv {
 	/* usual IV format */
 	struct {
-		/* logical block number within the file */
-		__le64 block_number;
+		/* data unit index within the file */
+		__le64 data_unit_index;
 
 		/* per-file nonce; only set in DIRECT_KEY mode */
 		u8 nonce[FILE_NONCE_SIZE];
 	};
 	/* IV format for IV_INO_LBLK_* modes */
 	struct {
 		/*
-		 * IV_INO_LBLK_64: logical block number within the file
-		 * IV_INO_LBLK_32: hashed inode number + logical block number
-		 *		   within the file, mod 2^32
+		 * IV_INO_LBLK_64: data unit index within the file
+		 * IV_INO_LBLK_32: hashed inode number + data unit index within
+		 *		   the file, mod 2^32
 		 */
-		__le32 block_number32;
+		__le32 data_unit_index32;
 
 		/* IV_INO_LBLK_64: inode number */
 		__le32 inode_number;
 	};
 	/* Any extra bytes up to the algorithm's IV size must be zeroed */
 	u8 bytes[MAX_IV_SIZE];
 };
 
 static void crypt_loop(const struct fscrypt_cipher *cipher, const u8 *key,
 		       union fscrypt_iv *iv, bool decrypting,
-		       size_t block_size, size_t padding, bool is_bnum_32bit)
+		       size_t data_unit_size, size_t padding,
+		       bool is_data_unit_index_32bit)
 {
-	u8 *buf = xmalloc(block_size);
+	u8 *buf = xmalloc(data_unit_size);
 	size_t res;
 
-	while ((res = xread(STDIN_FILENO, buf, block_size)) > 0) {
-		size_t crypt_len = block_size;
+	while ((res = xread(STDIN_FILENO, buf, data_unit_size)) > 0) {
+		size_t crypt_len = data_unit_size;
 
 		if (padding > 0) {
 			crypt_len = MAX(res, cipher->min_input_size);
 			crypt_len = ROUND_UP(crypt_len, padding);
-			crypt_len = MIN(crypt_len, block_size);
+			crypt_len = MIN(crypt_len, data_unit_size);
 		}
 		ASSERT(crypt_len >= res);
 		memset(&buf[res], 0, crypt_len - res);
 
 		if (decrypting)
 			cipher->decrypt(key, iv->bytes, buf, buf, crypt_len);
 		else
 			cipher->encrypt(key, iv->bytes, buf, buf, crypt_len);
 
 		full_write(STDOUT_FILENO, buf, crypt_len);
 
-		if (is_bnum_32bit)
-			iv->block_number32 = cpu_to_le32(
-					le32_to_cpu(iv->block_number32) + 1);
+		if (is_data_unit_index_32bit)
+			iv->data_unit_index32 = cpu_to_le32(
+				le32_to_cpu(iv->data_unit_index32) + 1);
 		else
-			iv->block_number = cpu_to_le64(
-					le64_to_cpu(iv->block_number) + 1);
+			iv->data_unit_index = cpu_to_le64(
+				le64_to_cpu(iv->data_unit_index) + 1);
 	}
 	free(buf);
 }
 
 /* The supported key derivation functions */
 enum kdf_algorithm {
 	KDF_NONE,
 	KDF_AES_128_ECB,
 	KDF_HKDF_SHA512,
 };
@@ -2063,21 +2064,21 @@ static u8 parse_mode_number(const char *arg)
 struct key_and_iv_params {
 	u8 master_key[MAX_KEY_SIZE];
 	int master_key_size;
 	enum kdf_algorithm kdf;
 	u8 mode_num;
 	u8 file_nonce[FILE_NONCE_SIZE];
 	bool file_nonce_specified;
 	bool direct_key;
 	bool iv_ino_lblk_64;
 	bool iv_ino_lblk_32;
-	u64 block_number;
+	u64 data_unit_index;
 	u64 inode_number;
 	u8 fs_uuid[UUID_SIZE];
 	bool fs_uuid_specified;
 };
 
 #define HKDF_CONTEXT_KEY_IDENTIFIER	1
 #define HKDF_CONTEXT_PER_FILE_ENC_KEY	2
 #define HKDF_CONTEXT_DIRECT_KEY		3
 #define HKDF_CONTEXT_IV_INO_LBLK_64_KEY	4
 #define HKDF_CONTEXT_DIRHASH_KEY	5
@@ -2171,40 +2172,40 @@ static void derive_real_key(const struct key_and_iv_params *params,
 	}
 }
 
 static void generate_iv(const struct key_and_iv_params *params,
 			union fscrypt_iv *iv)
 {
 	memset(iv, 0, sizeof(*iv));
 	if (params->direct_key) {
 		if (!params->file_nonce_specified)
 			die("--direct-key requires --file-nonce");
-		iv->block_number = cpu_to_le64(params->block_number);
+		iv->data_unit_index = cpu_to_le64(params->data_unit_index);
 		memcpy(iv->nonce, params->file_nonce, FILE_NONCE_SIZE);
 	} else if (params->iv_ino_lblk_64) {
-		if (params->block_number > UINT32_MAX)
-			die("iv-ino-lblk-64 can't use --block-number > UINT32_MAX");
+		if (params->data_unit_index > UINT32_MAX)
+			die("iv-ino-lblk-64 can't use --data-unit-index > UINT32_MAX");
 		if (params->inode_number == 0)
 			die("iv-ino-lblk-64 requires --inode-number");
 		if (params->inode_number > UINT32_MAX)
 			die("iv-ino-lblk-64 can't use --inode-number > UINT32_MAX");
-		iv->block_number32 = cpu_to_le32(params->block_number);
+		iv->data_unit_index32 = cpu_to_le32(params->data_unit_index);
 		iv->inode_number = cpu_to_le32(params->inode_number);
 	} else if (params->iv_ino_lblk_32) {
-		if (params->block_number > UINT32_MAX)
-			die("iv-ino-lblk-32 can't use --block-number > UINT32_MAX");
+		if (params->data_unit_index > UINT32_MAX)
+			die("iv-ino-lblk-32 can't use --data-unit-index > UINT32_MAX");
 		if (params->inode_number == 0)
 			die("iv-ino-lblk-32 requires --inode-number");
-		iv->block_number32 = cpu_to_le32(hash_inode_number(params) +
-						 params->block_number);
+		iv->data_unit_index32 = cpu_to_le32(hash_inode_number(params) +
+						    params->data_unit_index);
 	} else {
-		iv->block_number = cpu_to_le64(params->block_number);
+		iv->data_unit_index = cpu_to_le64(params->data_unit_index);
 	}
 }
 
 /*
  * Get the key and starting IV with which the encryption will actually be done.
  * If a KDF was specified, then a subkey is derived from the master key.
  * Otherwise, the master key is used directly.
  */
 static void get_key_and_iv(const struct key_and_iv_params *params,
 			   u8 *real_key, size_t real_key_size,
@@ -2245,57 +2246,57 @@ static void do_dump_key_identifier(const struct key_and_iv_params *params)
 
 static void parse_master_key(const char *arg, struct key_and_iv_params *params)
 {
 	params->master_key_size = hex2bin(arg, params->master_key,
 					  MAX_KEY_SIZE);
 	if (params->master_key_size < 0)
 		die("Invalid master_key: %s", arg);
 }
 
 enum {
-	OPT_BLOCK_NUMBER,
-	OPT_BLOCK_SIZE,
+	OPT_DATA_UNIT_INDEX,
+	OPT_DATA_UNIT_SIZE,
 	OPT_DECRYPT,
 	OPT_DIRECT_KEY,
 	OPT_DUMP_KEY_IDENTIFIER,
 	OPT_FILE_NONCE,
 	OPT_FS_UUID,
 	OPT_HELP,
 	OPT_INODE_NUMBER,
 	OPT_IV_INO_LBLK_32,
 	OPT_IV_INO_LBLK_64,
 	OPT_KDF,
 	OPT_MODE_NUM,
 	OPT_PADDING,
 };
 
 static const struct option longopts[] = {
-	{ "block-number",    required_argument, NULL, OPT_BLOCK_NUMBER },
-	{ "block-size",      required_argument, NULL, OPT_BLOCK_SIZE },
+	{ "data-unit-index", required_argument, NULL, OPT_DATA_UNIT_INDEX },
+	{ "data-unit-size",  required_argument, NULL, OPT_DATA_UNIT_SIZE },
 	{ "decrypt",         no_argument,       NULL, OPT_DECRYPT },
 	{ "direct-key",      no_argument,       NULL, OPT_DIRECT_KEY },
 	{ "dump-key-identifier", no_argument,   NULL, OPT_DUMP_KEY_IDENTIFIER },
 	{ "file-nonce",      required_argument, NULL, OPT_FILE_NONCE },
 	{ "fs-uuid",         required_argument, NULL, OPT_FS_UUID },
 	{ "help",            no_argument,       NULL, OPT_HELP },
 	{ "inode-number",    required_argument, NULL, OPT_INODE_NUMBER },
 	{ "iv-ino-lblk-32",  no_argument,       NULL, OPT_IV_INO_LBLK_32 },
 	{ "iv-ino-lblk-64",  no_argument,       NULL, OPT_IV_INO_LBLK_64 },
 	{ "kdf",             required_argument, NULL, OPT_KDF },
 	{ "mode-num",        required_argument, NULL, OPT_MODE_NUM },
 	{ "padding",         required_argument, NULL, OPT_PADDING },
 	{ NULL, 0, NULL, 0 },
 };
 
 int main(int argc, char *argv[])
 {
-	size_t block_size = 4096;
+	size_t data_unit_size = 4096;
 	bool decrypting = false;
 	bool dump_key_identifier = false;
 	struct key_and_iv_params params;
 	size_t padding = 0;
 	const struct fscrypt_cipher *cipher;
 	u8 real_key[MAX_KEY_SIZE];
 	union fscrypt_iv iv;
 	char *tmp;
 	int c;
 
@@ -2308,31 +2309,31 @@ int main(int argc, char *argv[])
 	test_sha2();
 	test_hkdf_sha512();
 	test_aes_256_xts();
 	test_aes_256_cts_cbc();
 	test_adiantum();
 	test_aes_256_hctr2();
 #endif
 
 	while ((c = getopt_long(argc, argv, "", longopts, NULL)) != -1) {
 		switch (c) {
-		case OPT_BLOCK_NUMBER:
+		case OPT_DATA_UNIT_INDEX:
 			errno = 0;
-			params.block_number = strtoull(optarg, &tmp, 10);
+			params.data_unit_index = strtoull(optarg, &tmp, 10);
 			if (*tmp || errno)
-				die("Invalid block number: %s", optarg);
+				die("Invalid data unit index: %s", optarg);
 			break;
-		case OPT_BLOCK_SIZE:
+		case OPT_DATA_UNIT_SIZE:
 			errno = 0;
-			block_size = strtoul(optarg, &tmp, 10);
-			if (block_size <= 0 || *tmp || errno)
-				die("Invalid block size: %s", optarg);
+			data_unit_size = strtoul(optarg, &tmp, 10);
+			if (data_unit_size <= 0 || *tmp || errno)
+				die("Invalid data unit size: %s", optarg);
 			break;
 		case OPT_DECRYPT:
 			decrypting = true;
 			break;
 		case OPT_DIRECT_KEY:
 			params.direct_key = true;
 			break;
 		case OPT_DUMP_KEY_IDENTIFIER:
 			dump_key_identifier = true;
 			break;
@@ -2394,25 +2395,25 @@ int main(int argc, char *argv[])
 	}
 	if (argc != 2) {
 		usage(stderr);
 		return 2;
 	}
 
 	cipher = find_fscrypt_cipher(argv[0]);
 	if (cipher == NULL)
 		die("Unknown cipher: %s", argv[0]);
 
-	if (block_size < cipher->min_input_size)
-		die("Block size of %zu bytes is too small for cipher %s",
-		    block_size, cipher->name);
+	if (data_unit_size < cipher->min_input_size)
+		die("Data unit size of %zu bytes is too small for cipher %s",
+		    data_unit_size, cipher->name);
 
 	parse_master_key(argv[1], &params);
 
 	if (params.master_key_size < cipher->keysize)
 		die("Master key is too short for cipher %s", cipher->name);
 
 	get_key_and_iv(&params, real_key, cipher->keysize, &iv);
 
-	crypt_loop(cipher, real_key, &iv, decrypting, block_size, padding,
+	crypt_loop(cipher, real_key, &iv, decrypting, data_unit_size, padding,
 		   params.iv_ino_lblk_64 || params.iv_ino_lblk_32);
 	return 0;
 }
diff --git a/tests/f2fs/002 b/tests/f2fs/002
index 8235d88a..c0bf440b 100755
--- a/tests/f2fs/002
+++ b/tests/f2fs/002
@@ -133,30 +133,30 @@ nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode)
 
 echo -e "\n# Dumping the file's raw data"
 _dump_ciphertext_blocks $SCRATCH_DEV $blocklist > $tmp.raw
 
 echo -e "\n# Decrypting the file's data"
 TEST_RAW_KEY_HEX=$(echo "$TEST_RAW_KEY" | tr -d '\\x')
 decrypt_blocks()
 {
 	$here/src/fscrypt-crypt-util "$@"			\
 		--decrypt					\
-		--block-size=$block_size			\
+		--data-unit-size=$block_size			\
 		--file-nonce=$nonce				\
 		--kdf=HKDF-SHA512				\
 		AES-256-XTS					\
 		$TEST_RAW_KEY_HEX
 }
 head -c $num_compressible_bytes $tmp.raw \
-	| decrypt_blocks --block-number=1 > $tmp.decrypted
+	| decrypt_blocks --data-unit-index=1 > $tmp.decrypted
 dd if=$tmp.raw bs=$cluster_bytes skip=$num_compressible_clusters status=none \
-	| decrypt_blocks --block-number=$num_compressible_blocks \
+	| decrypt_blocks --data-unit-index=$num_compressible_blocks \
 	>> $tmp.decrypted
 
 # Decompress the compressed clusters using the lz4 command-line tool.
 #
 # Each f2fs compressed cluster begins with a 24-byte header, starting with the
 # compressed size in bytes (excluding the header) as a __le32.  The header is
 # followed by the actual compressed data; for LZ4, that means an LZ4 block.
 #
 # Unfortunately, the lz4 command-line tool only deals with LZ4 *frames*
 # (https://github.com/lz4/lz4/blob/master/doc/lz4_Frame_format.md) and can't
-- 
2.42.1

[PATCH v2 2/4] common/rc: fix _require_xfs_io_command with digits in argument

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

'_require_xfs_io_command set_encpolicy -s' does not work as expected
because the following in the output of 'xfs_io -c "help set_encpolicy"':

     -s LOG2_DUSIZE -- log2 of data unit size

... does not match the regex:

    "^ -s ([a-zA-Z_]+ )?--"

... because the 2 in the argument name LOG2_DUSIZE is not matched.  Fix
the regex to support digits in the argument name.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 common/rc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/common/rc b/common/rc
index cc92fe06..dab672d8 100644
--- a/common/rc
+++ b/common/rc
@@ -2719,21 +2719,21 @@ _require_xfs_io_command()
 	echo $testio | grep -q "foreign file active" && \
 		_notrun "xfs_io $command $param_checked not supported on $FSTYP"
 	echo $testio | grep -q "Function not implemented" && \
 		_notrun "xfs_io $command $param_checked support is missing (missing syscall?)"
 	echo $testio | grep -q "unknown flag" && \
 		_notrun "xfs_io $command $param_checked support is missing (unknown flag)"
 
 	[ -n "$param" ] || return
 
 	if [ -z "$param_checked" ]; then
-		$XFS_IO_PROG -c "help $command" | grep -E -q "^ $param ([a-zA-Z_]+ )?--" || \
+		$XFS_IO_PROG -c "help $command" | grep -E -q "^ $param ([a-zA-Z0-9_]+ )?--" || \
 			_notrun "xfs_io $command doesn't support $param"
 	else
 		# xfs_io could result in "command %c not supported" if it was
 		# built on kernels not supporting pwritev2() calls
 		echo $testio | grep -q "\(invalid option\|not supported\)" && \
 			_notrun "xfs_io $command doesn't support $param"
 	fi
 
 	# On XFS, ioctl(FSSETXATTR)(called by xfs_io -c "chattr") maskes off unsupported
 	# or invalid flags silently so need to check these flags by extra ioctl(FSGETXATTR)
-- 
2.42.1

[PATCH v2 3/4] common/encrypt: support custom data unit size

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Make _require_scratch_encryption() and
_require_encryption_policy_support() support the new '-s' option to
set_encpolicy to specify a custom value of log2_data_unit_size.

Likewise, make _verify_ciphertext_for_encryption_policy() accept an
argument "log2_dusize=*" to cause it to use the specified data unit size
for the test and verify that the file contents are encrypted as expected
for that data unit size.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 common/encrypt | 38 ++++++++++++++++++++++++++++++--------
 1 file changed, 30 insertions(+), 8 deletions(-)

diff --git a/common/encrypt b/common/encrypt
index 5688745c..d90a566a 100644
--- a/common/encrypt
+++ b/common/encrypt
@@ -1,32 +1,41 @@
 ##/bin/bash
 # SPDX-License-Identifier: GPL-2.0
 # Copyright (c) 2016 Google, Inc.  All Rights Reserved.
 #
 # Functions for setting up and testing file encryption
 
 #
 # _require_scratch_encryption [-c CONTENTS_MODE] [-n FILENAMES_MODE]
 #			      [-f POLICY_FLAGS] [-v POLICY_VERSION]
+#			      [-s LOG2_DUSIZE]
 #
 # Require encryption support on the scratch device.
 #
 # This checks for support for the default type of encryption policy (v1 with
 # AES-256-XTS and AES-256-CTS).  Options can be specified to also require
 # support for a different type of encryption policy.
 #
 _require_scratch_encryption()
 {
-	_require_scratch
+	local arg
 
+	_require_scratch
 	_require_xfs_io_command "set_encpolicy"
 
+	for arg; do
+		if [ "$arg" = "-s" ]; then
+			# -s option was added later.  Make sure it's available.
+			_require_xfs_io_command "set_encpolicy" "-s"
+		fi
+	done
+
 	# The 'test_dummy_encryption' mount option interferes with trying to use
 	# encryption for real, even if we are just trying to get/set policies
 	# and never put any keys in the keyring.  So skip the real encryption
 	# tests if the 'test_dummy_encryption' mount option was specified.
 	_exclude_scratch_mount_option "test_dummy_encryption"
 
 	# Make a filesystem on the scratch device with the encryption feature
 	# enabled.  If this fails then probably the userspace tools (e.g.
 	# e2fsprogs or f2fs-tools) are too old to understand encryption.
 	if ! _scratch_mkfs_encrypted &>>$seqres.full; then
@@ -67,35 +76,35 @@ _require_scratch_encryption()
 _require_encryption_policy_support()
 {
 	local mnt=$1
 	local dir=$mnt/tmpdir
 	local set_encpolicy_args=""
 	local policy_flags=0
 	local policy_version=1
 	local c
 
 	OPTIND=2
-	while getopts "c:n:f:v:" c; do
+	while getopts "c:n:f:s:v:" c; do
 		case $c in
-		c|n)
+		c|n|s)
 			set_encpolicy_args+=" -$c $OPTARG"
 			;;
 		f)
 			set_encpolicy_args+=" -$c $OPTARG"
 			policy_flags=$OPTARG
 			;;
 		v)
 			set_encpolicy_args+=" -$c $OPTARG"
 			policy_version=$OPTARG
 			;;
 		*)
-			_fail "Unrecognized option '$c'"
+			_fail "${FUNCNAME[0]}: unrecognized option '$c'"
 			;;
 		esac
 	done
 	set_encpolicy_args=${set_encpolicy_args# }
 
 	echo "Checking whether kernel supports encryption policy: $set_encpolicy_args" \
 		>> $seqres.full
 
 	if (( policy_flags & (FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64 |
 			      FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32) )); then
@@ -756,28 +765,27 @@ _do_verify_ciphertext_for_encryption_policy()
 
 	# Now unmount the filesystem and verify the ciphertext we just wrote.
 	_scratch_unmount
 
 	echo "Verifying encrypted file contents" >> $seqres.full
 	for f in "${test_contents_files[@]}"; do
 		read -r src inode blocklist <<< "$f"
 		nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode)
 		_dump_ciphertext_blocks $SCRATCH_DEV $blocklist > $tmp.actual_contents
 		$crypt_contents_cmd $contents_encryption_mode $raw_key_hex \
-			--file-nonce=$nonce --data-unit-size=$blocksize \
-			--inode-number=$inode < $src > $tmp.expected_contents
+			--file-nonce=$nonce --inode-number=$inode \
+                        < $src > $tmp.expected_contents
 		if ! cmp $tmp.expected_contents $tmp.actual_contents; then
 			_fail "Expected encrypted contents != actual encrypted contents.  File: $f"
 		fi
 		$crypt_contents_cmd $contents_encryption_mode $raw_key_hex \
-			--decrypt --file-nonce=$nonce \
-                        --data-unit-size=$blocksize --inode-number=$inode \
+			--decrypt --file-nonce=$nonce --inode-number=$inode \
 			< $tmp.actual_contents > $tmp.decrypted_contents
 		if ! cmp $src $tmp.decrypted_contents; then
 			_fail "Contents decryption sanity check failed.  File: $f"
 		fi
 	done
 
 	echo "Verifying encrypted file names" >> $seqres.full
 	for f in "${test_filenames_files[@]}"; do
 		read -r name inode dir_inode padding <<< "$f"
 		nonce=$(_get_encryption_nonce $SCRATCH_DEV $dir_inode)
@@ -837,28 +845,30 @@ _fscrypt_mode_name_to_num()
 # policy of the specified type is used.
 #
 # The first two parameters are the contents and filenames encryption modes to
 # test.  The following optional parameters are also accepted to further modify
 # the type of encryption policy that is tested:
 #
 #	'v2':			test a v2 encryption policy
 #	'direct':		test the DIRECT_KEY policy flag
 #	'iv_ino_lblk_64':	test the IV_INO_LBLK_64 policy flag
 #	'iv_ino_lblk_32':	test the IV_INO_LBLK_32 policy flag
+#	'log2_dusize=N':        test the log2_data_unit_size field
 #
 _verify_ciphertext_for_encryption_policy()
 {
 	local contents_encryption_mode=$1
 	local filenames_encryption_mode=$2
 	local opt
 	local policy_version=1
 	local policy_flags=0
+	local log2_dusize=0
 	local set_encpolicy_args=""
 	local crypt_util_args=""
 	local crypt_util_contents_args=""
 	local crypt_util_filename_args=""
 	local expected_identifier
 
 	shift 2
 	for opt; do
 		case "$opt" in
 		v2)
@@ -870,30 +880,36 @@ _verify_ciphertext_for_encryption_policy()
 				_fail "For direct key mode, contents and filenames modes must match"
 			fi
 			(( policy_flags |= FSCRYPT_POLICY_FLAG_DIRECT_KEY ))
 			;;
 		iv_ino_lblk_64)
 			(( policy_flags |= FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64 ))
 			;;
 		iv_ino_lblk_32)
 			(( policy_flags |= FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32 ))
 			;;
+		log2_dusize=*)
+			log2_dusize=$(echo "$opt" | sed 's/^log2_dusize=//')
+			;;
 		*)
 			_fail "Unknown option '$opt' passed to ${FUNCNAME[0]}"
 			;;
 		esac
 	done
 	local contents_mode_num=$(_fscrypt_mode_name_to_num $contents_encryption_mode)
 	local filenames_mode_num=$(_fscrypt_mode_name_to_num $filenames_encryption_mode)
 
 	set_encpolicy_args+=" -c $contents_mode_num"
 	set_encpolicy_args+=" -n $filenames_mode_num"
+	if (( log2_dusize != 0 )); then
+		set_encpolicy_args+=" -s $log2_dusize"
+	fi
 	crypt_util_contents_args+=" --mode-num=$contents_mode_num"
 	crypt_util_filename_args+=" --mode-num=$filenames_mode_num"
 
 	if (( policy_version > 1 )); then
 		set_encpolicy_args+=" -v 2"
 		crypt_util_args+=" --kdf=HKDF-SHA512"
 		if (( policy_flags & FSCRYPT_POLICY_FLAG_DIRECT_KEY )); then
 			crypt_util_args+=" --direct-key"
 		elif (( policy_flags & FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64 )); then
 			crypt_util_args+=" --iv-ino-lblk-64"
@@ -923,20 +939,26 @@ _verify_ciphertext_for_encryption_policy()
 
 	echo "Creating encryption-capable filesystem" >> $seqres.full
 	if (( policy_flags & (FSCRYPT_POLICY_FLAG_IV_INO_LBLK_64 |
 			      FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32) )); then
 		_scratch_mkfs_stable_inodes_encrypted &>> $seqres.full
 	else
 		_scratch_mkfs_encrypted &>> $seqres.full
 	fi
 	_scratch_mount
 
+	if (( log2_dusize != 0 )); then
+		crypt_util_contents_args+=" --data-unit-size=$((1 << log2_dusize))"
+	else
+		crypt_util_contents_args+=" --data-unit-size=$(_get_block_size $SCRATCH_MNT)"
+	fi
+
 	crypt_util_args+=" --fs-uuid=$(blkid -s UUID -o value $SCRATCH_DEV | tr -d -)"
 
 	crypt_util_contents_args+="$crypt_util_args"
 	crypt_util_filename_args+="$crypt_util_args"
 
 	echo "Generating encryption key" >> $seqres.full
 	local raw_key=$(_generate_raw_encryption_key)
 	if (( policy_version > 1 )); then
 		local keyspec=$(_add_enckey $SCRATCH_MNT "$raw_key" \
 				| awk '{print $NF}')
-- 
2.42.1

[PATCH v2 4/4] generic: add test for custom crypto data unit size

[Eric Biggers]

From: Eric Biggers <ebiggers@google.com>

Add a test that verifies the on-disk format of encrypted files that use
a crypto data unit size that differs from the filesystem block size.
This tests the functionality that was introduced in Linux 6.7 by kernel
commit 5b1188847180 ("fscrypt: support crypto data unit size less than
filesystem block size").

This depends on the xfsprogs patch
"xfs_io/encrypt: support specifying crypto data unit size"
(https://lore.kernel.org/r/20231013062639.141468-1-ebiggers@kernel.org)
which adds the '-s' option to the set_encpolicy command of xfs_io.

As usual, the test skips itself when any prerequisite isn't met.

Signed-off-by: Eric Biggers <ebiggers@google.com>
---
 tests/generic/900     | 29 +++++++++++++++++++++++++++++
 tests/generic/900.out | 11 +++++++++++
 2 files changed, 40 insertions(+)
 create mode 100755 tests/generic/900
 create mode 100644 tests/generic/900.out

diff --git a/tests/generic/900 b/tests/generic/900
new file mode 100755
index 00000000..8d1b5766
--- /dev/null
+++ b/tests/generic/900
@@ -0,0 +1,29 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright 2023 Google LLC
+#
+# FS QA Test No. generic/900
+#
+# Verify the on-disk format of encrypted files that use a crypto data unit size
+# that differs from the filesystem block size.  This tests the functionality
+# that was introduced in Linux 6.7 by kernel commit 5b1188847180
+# ("fscrypt: support crypto data unit size less than filesystem block size").
+#
+. ./common/preamble
+_begin_fstest auto quick encrypt
+
+. ./common/filter
+. ./common/encrypt
+
+_supported_fs generic
+
+# For now, just test 512-byte and 1024-byte data units.  Filesystems accept
+# power-of-2 sizes between 512 and the filesystem block size, inclusively.
+# Testing 512 and 1024 ensures this test will run for any FS block size >= 1024
+# (provided that the filesystem supports sub-block data units at all).
+_verify_ciphertext_for_encryption_policy AES-256-XTS AES-256-CTS-CBC v2 log2_dusize=9
+_verify_ciphertext_for_encryption_policy AES-256-XTS AES-256-CTS-CBC v2 log2_dusize=10
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/900.out b/tests/generic/900.out
new file mode 100644
index 00000000..3259f08c
--- /dev/null
+++ b/tests/generic/900.out
@@ -0,0 +1,11 @@
+QA output created by 900
+
+Verifying ciphertext with parameters:
+	contents_encryption_mode: AES-256-XTS
+	filenames_encryption_mode: AES-256-CTS-CBC
+	options: v2 log2_dusize=9
+
+Verifying ciphertext with parameters:
+	contents_encryption_mode: AES-256-XTS
+	filenames_encryption_mode: AES-256-CTS-CBC
+	options: v2 log2_dusize=10
-- 
2.42.1

Re: [PATCH v2 4/4] generic: add test for custom crypto data unit size

[Zorro Lang]

On Tue, Nov 21, 2023 at 02:39:09PM -0800, Eric Biggers wrote:
> From: Eric Biggers <ebiggers@google.com>
> 
> Add a test that verifies the on-disk format of encrypted files that use
> a crypto data unit size that differs from the filesystem block size.
> This tests the functionality that was introduced in Linux 6.7 by kernel
> commit 5b1188847180 ("fscrypt: support crypto data unit size less than
> filesystem block size").
> 
> This depends on the xfsprogs patch
> "xfs_io/encrypt: support specifying crypto data unit size"
> (https://lore.kernel.org/r/20231013062639.141468-1-ebiggers@kernel.org)
> which adds the '-s' option to the set_encpolicy command of xfs_io.
> 
> As usual, the test skips itself when any prerequisite isn't met.
> 
> Signed-off-by: Eric Biggers <ebiggers@google.com>
> ---
>  tests/generic/900     | 29 +++++++++++++++++++++++++++++
>  tests/generic/900.out | 11 +++++++++++
>  2 files changed, 40 insertions(+)
>  create mode 100755 tests/generic/900
>  create mode 100644 tests/generic/900.out
> 
> diff --git a/tests/generic/900 b/tests/generic/900
> new file mode 100755
> index 00000000..8d1b5766
> --- /dev/null
> +++ b/tests/generic/900
> @@ -0,0 +1,29 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright 2023 Google LLC
> +#
> +# FS QA Test No. generic/900
> +#
> +# Verify the on-disk format of encrypted files that use a crypto data unit size
> +# that differs from the filesystem block size.  This tests the functionality
> +# that was introduced in Linux 6.7 by kernel commit 5b1188847180
> +# ("fscrypt: support crypto data unit size less than filesystem block size").

I'll write this part as:

  _wants_kernel_commit 5b1188847180
	"fscrypt: support crypto data unit size less than filesystem block size"

when I merge it.

> +#
> +. ./common/preamble
> +_begin_fstest auto quick encrypt
> +
> +. ./common/filter
> +. ./common/encrypt
> +
> +_supported_fs generic
> +
> +# For now, just test 512-byte and 1024-byte data units.  Filesystems accept
> +# power-of-2 sizes between 512 and the filesystem block size, inclusively.
> +# Testing 512 and 1024 ensures this test will run for any FS block size >= 1024
> +# (provided that the filesystem supports sub-block data units at all).
> +_verify_ciphertext_for_encryption_policy AES-256-XTS AES-256-CTS-CBC v2 log2_dusize=9

Oh, all _require_scratch_... things are in this helper. I was wondering
why it doesn't need to _require_scratch :)

This patchset looks good to me, tested passed on latest upstream kernel.
Thanks for this upgrade. Feel free to ping me, if your patchset be blocked
long time.

Reviewed-by: Zorro Lang <zlang@redhat.com>

> +_verify_ciphertext_for_encryption_policy AES-256-XTS AES-256-CTS-CBC v2 log2_dusize=10
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/900.out b/tests/generic/900.out
> new file mode 100644
> index 00000000..3259f08c
> --- /dev/null
> +++ b/tests/generic/900.out
> @@ -0,0 +1,11 @@
> +QA output created by 900
> +
> +Verifying ciphertext with parameters:
> +	contents_encryption_mode: AES-256-XTS
> +	filenames_encryption_mode: AES-256-CTS-CBC
> +	options: v2 log2_dusize=9
> +
> +Verifying ciphertext with parameters:
> +	contents_encryption_mode: AES-256-XTS
> +	filenames_encryption_mode: AES-256-CTS-CBC
> +	options: v2 log2_dusize=10
> -- 
> 2.42.1
> 
> 

Re: [PATCH v2 0/4] xfstests: test custom crypto data unit size

[Eric Biggers]

On Tue, Nov 21, 2023 at 02:39:05PM -0800, Eric Biggers wrote:
> This series adds a test that verifies the on-disk format of encrypted
> files that use a crypto data unit size that differs from the filesystem
> block size.  This tests the functionality that was introduced in Linux
> 6.7 by kernel commit 5b1188847180 ("fscrypt: support crypto data unit
> size less than filesystem block size").

Hi Zorro, can you consider applying this series?  It's been out for review for
3 months, so I don't think reviews are going to come in.  The prerequisite
xfsprogs patch is already present on the for-next branch of xfsprogs.

Thanks!

- Eric

Re: [PATCH v2 0/4] xfstests: test custom crypto data unit size

[Zorro Lang]

On Wed, Jan 10, 2024 at 07:54:44PM -0800, Eric Biggers wrote:
> On Tue, Nov 21, 2023 at 02:39:05PM -0800, Eric Biggers wrote:
> > This series adds a test that verifies the on-disk format of encrypted
> > files that use a crypto data unit size that differs from the filesystem
> > block size.  This tests the functionality that was introduced in Linux
> > 6.7 by kernel commit 5b1188847180 ("fscrypt: support crypto data unit
> > size less than filesystem block size").
> 
> Hi Zorro, can you consider applying this series?  It's been out for review for
> 3 months, so I don't think reviews are going to come in.  The prerequisite
> xfsprogs patch is already present on the for-next branch of xfsprogs.

Sure Eric, I thought about this patchset last week, I saw that xfsprogs
patchset has been merge. I'll give this patchset a basic test, to make
sure it won't break xfstests, then merge it.

Thanks, and sorry this late.
Zorro

> 
> Thanks!
> 
> - Eric
>