* [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames
@ 2019-04-26 20:41 Eric Biggers
2019-04-26 20:41 ` [RFC PATCH 1/7] common/encrypt: introduce helpers for set_encpolicy and get_encpolicy Eric Biggers
` (7 more replies)
0 siblings, 8 replies; 14+ messages in thread
From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw)
To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel
Hello,
This series adds xfstests which verify that encrypted contents and
filenames on ext4 and f2fs are actually correct, i.e. that the
encryption uses the correct algorithms, keys, IVs, and padding amounts.
The new tests work by creating encrypted files, unmounting the
filesystem, reading the ciphertext from disk using dd and debugfs or
dump.f2fs, and then comparing it against ciphertext computed
independently by a new test program that implements the same algorithms.
These tests are important because:
- The whole point of file encryption is that the files are actually
encrypted correctly on-disk. Except for generic/399, current xfstests
only tests the filesystem semantics, not the actual encryption.
generic/399 only tests for incompressibility of encrypted file
contents using one particular encryption setting, which isn't much.
- fscrypt now supports 4 main combinations of encryption settings,
rather than 1 as it did originally. This may be doubled to 8 soon
(https://patchwork.kernel.org/patch/10908153/). We should test all
settings. And without tests, even if the initial implementation is
correct, breakage in one specific setting could go undetected.
- Though Linux's crypto API has self-tests, these only test the
algorithms themselves, not how they are used, e.g. by fscrypt.
Patch 1 is a cleanup patch. Patches 2-4 add the common helpers for
ciphertext verification tests. Patches 5-7 add the actual tests.
These tests require e2fsprogs and f2fs-tools patches I recently sent out
to fix printing encrypted filenames. So, this series might not be
suitable for merging into mainline xfstests until those patches are
applied. Regardless, comments are appreciated. The needed patches are:
debugfs: avoid ambiguity when printing filenames (https://marc.info/?l=linux-ext4&m=155596495624232&w=2)
f2fs-tools: improve filename printing (https://sourceforge.net/p/linux-f2fs/mailman/message/36648641/)
This series can also be retrieved from git at
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/xfstests-dev.git
branch "ciphertext-verification".
I also have patches on top of this series which verify the ciphertext
produced from v2 encryption policies, which are proposed by my kernel
patch series "fscrypt: key management improvements"
(https://patchwork.kernel.org/cover/10908107/). v2 encryption policies
will use a different key derivation function, and thus their ciphertext
will be different. These additional patches can be found at branch
"fscrypt-key-mgmt-improvements" of my git repo above. But I've arranged
things such that this shorter series can potentially be applied earlier,
to test what's in the kernel now.
Eric Biggers (7):
common/encrypt: introduce helpers for set_encpolicy and get_encpolicy
fscrypt-crypt-util: add utility for reproducing fscrypt encrypted data
common/encrypt: support requiring other encryption settings
common/encrypt: add helper for ciphertext verification tests
generic: verify ciphertext of v1 encryption policies with AES-256
generic: verify ciphertext of v1 encryption policies with AES-128
generic: verify ciphertext of v1 encryption policies with Adiantum
.gitignore | 1 +
common/encrypt | 482 ++++++++++-
src/Makefile | 3 +-
src/fscrypt-crypt-util.c | 1645 ++++++++++++++++++++++++++++++++++++++
tests/ext4/024 | 3 +-
tests/generic/395 | 28 +-
tests/generic/395.out | 2 +-
tests/generic/396 | 15 +-
tests/generic/397 | 3 +-
tests/generic/398 | 5 +-
tests/generic/399 | 3 +-
tests/generic/419 | 3 +-
tests/generic/421 | 3 +-
tests/generic/429 | 3 +-
tests/generic/435 | 3 +-
tests/generic/440 | 5 +-
tests/generic/700 | 41 +
tests/generic/700.out | 5 +
tests/generic/701 | 41 +
tests/generic/701.out | 5 +
tests/generic/702 | 43 +
tests/generic/702.out | 10 +
tests/generic/group | 3 +
23 files changed, 2308 insertions(+), 47 deletions(-)
create mode 100644 src/fscrypt-crypt-util.c
create mode 100755 tests/generic/700
create mode 100644 tests/generic/700.out
create mode 100755 tests/generic/701
create mode 100644 tests/generic/701.out
create mode 100755 tests/generic/702
create mode 100644 tests/generic/702.out
--
2.21.0.593.g511ec345e18-goog
^ permalink raw reply [flat|nested] 14+ messages in thread* [RFC PATCH 1/7] common/encrypt: introduce helpers for set_encpolicy and get_encpolicy 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers @ 2019-04-26 20:41 ` Eric Biggers 2019-05-12 12:21 ` Eryu Guan 2019-04-26 20:41 ` [RFC PATCH 2/7] fscrypt-crypt-util: add utility for reproducing fscrypt encrypted data Eric Biggers ` (6 subsequent siblings) 7 siblings, 1 reply; 14+ messages in thread From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel From: Eric Biggers <ebiggers@google.com> For conciseness in tests, add helper functions that wrap the xfs_io commands 'set_encpolicy' and 'get_encpolicy'. Then update all encryption tests to use them. Signed-off-by: Eric Biggers <ebiggers@google.com> --- common/encrypt | 34 ++++++++++++++++++++++++++++++++-- tests/ext4/024 | 3 +-- tests/generic/395 | 28 +++++++++++++--------------- tests/generic/395.out | 2 +- tests/generic/396 | 15 +++++++-------- tests/generic/397 | 3 +-- tests/generic/398 | 5 ++--- tests/generic/399 | 3 +-- tests/generic/419 | 3 +-- tests/generic/421 | 3 +-- tests/generic/429 | 3 +-- tests/generic/435 | 3 +-- tests/generic/440 | 5 ++--- 13 files changed, 64 insertions(+), 46 deletions(-) diff --git a/common/encrypt b/common/encrypt index 1b10aa71..54d873fa 100644 --- a/common/encrypt +++ b/common/encrypt @@ -38,8 +38,7 @@ _require_scratch_encryption() # presence of /sys/fs/ext4/features/encryption, but this is broken on # some older kernels and is ext4-specific anyway.) mkdir $SCRATCH_MNT/tmpdir - if $XFS_IO_PROG -c set_encpolicy $SCRATCH_MNT/tmpdir \ - 2>&1 >>$seqres.full | \ + if _set_encpolicy $SCRATCH_MNT/tmpdir 2>&1 >>$seqres.full | \ egrep -q 'Inappropriate ioctl for device|Operation not supported' then _notrun "kernel does not support $FSTYP encryption" @@ -175,3 +174,34 @@ _revoke_encryption_key() local keyid=$($KEYCTL_PROG search @s logon $FSTYP:$keydesc) $KEYCTL_PROG revoke $keyid >>$seqres.full } + +# Set an encryption policy on the specified directory. +_set_encpolicy() +{ + local dir=$1 + shift + + $XFS_IO_PROG -c "set_encpolicy $*" "$dir" +} + +_user_do_set_encpolicy() +{ + local dir=$1 + shift + + _user_do "$XFS_IO_PROG -c \"set_encpolicy $*\" \"$dir\"" +} + +_require_get_encpolicy() +{ + _require_xfs_io_command "get_encpolicy" +} + +# Display the specified file or directory's encryption policy. +_get_encpolicy() +{ + local file=$1 + shift + + $XFS_IO_PROG -c "get_encpolicy $*" "$file" +} diff --git a/tests/ext4/024 b/tests/ext4/024 index c54d2519..a86cc417 100755 --- a/tests/ext4/024 +++ b/tests/ext4/024 @@ -33,7 +33,6 @@ rm -f $seqres.full _supported_fs ext4 _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$KEYCTL_PROG" keyctl _new_session_keyring @@ -55,7 +54,7 @@ _scratch_mkfs_encrypted &>>$seqres.full _scratch_mount mkdir $SCRATCH_MNT/edir keydesc=$(_generate_encryption_key) -$XFS_IO_PROG -c "set_encpolicy $keydesc" $SCRATCH_MNT/edir +_set_encpolicy $SCRATCH_MNT/edir $keydesc echo foo > $SCRATCH_MNT/edir/file inum=$(stat -c '%i' $SCRATCH_MNT/edir/file) _scratch_unmount diff --git a/tests/generic/395 b/tests/generic/395 index cae97cef..cdb348c1 100755 --- a/tests/generic/395 +++ b/tests/generic/395 @@ -33,8 +33,7 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "get_encpolicy" -_require_xfs_io_command "set_encpolicy" +_require_get_encpolicy _require_user _scratch_mkfs_encrypted &>> $seqres.full @@ -45,7 +44,7 @@ check_no_policy() # When a file is unencrypted, FS_IOC_GET_ENCRYPTION_POLICY currently # fails with ENOENT on ext4 but with ENODATA on f2fs. TODO: it's # planned to consistently use ENODATA. For now this test accepts both. - $XFS_IO_PROG -c "get_encpolicy" $1 |& + _get_encpolicy $1 |& sed -e 's/No such file or directory/No data available/' } @@ -54,25 +53,25 @@ empty_dir=$SCRATCH_MNT/empty_dir echo -e "\n*** Setting encryption policy on empty directory ***" mkdir $empty_dir check_no_policy $empty_dir |& _filter_scratch -$XFS_IO_PROG -c "set_encpolicy 0000111122223333" $empty_dir -$XFS_IO_PROG -c "get_encpolicy" $empty_dir | _filter_scratch +_set_encpolicy $empty_dir 0000111122223333 +_get_encpolicy $empty_dir | _filter_scratch # Should be able to set the same policy again, but not a different one. # TODO: the error code for "already has a different policy" is planned to switch # from EINVAL to EEXIST. For now this test accepts both. echo -e "\n*** Setting encryption policy again ***" -$XFS_IO_PROG -c "set_encpolicy 0000111122223333" $empty_dir -$XFS_IO_PROG -c "get_encpolicy" $empty_dir | _filter_scratch -$XFS_IO_PROG -c "set_encpolicy 4444555566667777" $empty_dir |& \ +_set_encpolicy $empty_dir 0000111122223333 +_get_encpolicy $empty_dir | _filter_scratch +_set_encpolicy $empty_dir 4444555566667777 |& \ _filter_scratch | sed -e 's/Invalid argument/File exists/' -$XFS_IO_PROG -c "get_encpolicy" $empty_dir | _filter_scratch +_get_encpolicy $empty_dir | _filter_scratch # Should *not* be able to set an encryption policy on a nonempty directory nonempty_dir=$SCRATCH_MNT/nonempty_dir echo -e "\n*** Setting encryption policy on nonempty directory ***" mkdir $nonempty_dir touch $nonempty_dir/file -$XFS_IO_PROG -c "set_encpolicy" $nonempty_dir |& _filter_scratch +_set_encpolicy $nonempty_dir |& _filter_scratch check_no_policy $nonempty_dir |& _filter_scratch # Should *not* be able to set an encryption policy on a nondirectory file, even @@ -83,7 +82,7 @@ check_no_policy $nonempty_dir |& _filter_scratch nondirectory=$SCRATCH_MNT/nondirectory echo -e "\n*** Setting encryption policy on nondirectory ***" touch $nondirectory -$XFS_IO_PROG -c "set_encpolicy" $nondirectory |& \ +_set_encpolicy $nondirectory |& \ _filter_scratch | sed -e 's/Invalid argument/Not a directory/' check_no_policy $nondirectory |& _filter_scratch @@ -93,8 +92,7 @@ check_no_policy $nondirectory |& _filter_scratch unauthorized_dir=$SCRATCH_MNT/unauthorized_dir echo -e "\n*** Setting encryption policy on another user's directory ***" mkdir $unauthorized_dir -su $qa_user -c "$XFS_IO_PROG -c \"set_encpolicy\" $unauthorized_dir" |& \ - _filter_scratch +_user_do_set_encpolicy $unauthorized_dir |& _filter_scratch check_no_policy $unauthorized_dir |& _filter_scratch # Should *not* be able to set an encryption policy on a directory on a @@ -104,12 +102,12 @@ check_no_policy $unauthorized_dir |& _filter_scratch echo -e "\n*** Setting encryption policy on readonly filesystem ***" mkdir $SCRATCH_MNT/ro_dir $SCRATCH_MNT/ro_bind_mnt _scratch_remount ro -$XFS_IO_PROG -c "set_encpolicy" $SCRATCH_MNT/ro_dir |& _filter_scratch +_set_encpolicy $SCRATCH_MNT/ro_dir |& _filter_scratch check_no_policy $SCRATCH_MNT/ro_dir |& _filter_scratch _scratch_remount rw mount --bind $SCRATCH_MNT $SCRATCH_MNT/ro_bind_mnt mount -o remount,ro,bind $SCRATCH_MNT/ro_bind_mnt -$XFS_IO_PROG -c "set_encpolicy" $SCRATCH_MNT/ro_bind_mnt/ro_dir |& _filter_scratch +_set_encpolicy $SCRATCH_MNT/ro_bind_mnt/ro_dir |& _filter_scratch check_no_policy $SCRATCH_MNT/ro_bind_mnt/ro_dir |& _filter_scratch umount $SCRATCH_MNT/ro_bind_mnt diff --git a/tests/generic/395.out b/tests/generic/395.out index 90c450f2..2c55d7a9 100644 --- a/tests/generic/395.out +++ b/tests/generic/395.out @@ -33,7 +33,7 @@ SCRATCH_MNT/nondirectory: failed to set encryption policy: Not a directory SCRATCH_MNT/nondirectory: failed to get encryption policy: No data available *** Setting encryption policy on another user's directory *** -SCRATCH_MNT/unauthorized_dir: failed to set encryption policy: Permission denied +Permission denied SCRATCH_MNT/unauthorized_dir: failed to get encryption policy: No data available *** Setting encryption policy on readonly filesystem *** diff --git a/tests/generic/396 b/tests/generic/396 index 50c2c2e0..7027c339 100755 --- a/tests/generic/396 +++ b/tests/generic/396 @@ -34,7 +34,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _scratch_mkfs_encrypted &>> $seqres.full _scratch_mount @@ -42,23 +41,23 @@ dir=$SCRATCH_MNT/dir mkdir $dir echo -e "\n*** Invalid contents encryption mode ***" -$XFS_IO_PROG -c "set_encpolicy -c 0xFF" $dir |& _filter_scratch +_set_encpolicy $dir -c 0xFF |& _filter_scratch echo -e "\n*** Invalid filenames encryption mode ***" -$XFS_IO_PROG -c "set_encpolicy -n 0xFF" $dir |& _filter_scratch +_set_encpolicy $dir -n 0xFF |& _filter_scratch echo -e "\n*** Invalid flags ***" -$XFS_IO_PROG -c "set_encpolicy -f 0xFF" $dir |& _filter_scratch +_set_encpolicy $dir -f 0xFF |& _filter_scratch echo -e "\n*** Invalid policy version ***" -$XFS_IO_PROG -c "set_encpolicy -v 0xFF" $dir |& _filter_scratch +_set_encpolicy $dir -v 0xFF |& _filter_scratch # Currently, the only supported combination of modes is AES-256-XTS for contents # and AES-256-CTS for filenames. Nothing else should be accepted. echo -e "\n*** Invalid combinations of modes ***" -$XFS_IO_PROG -c "set_encpolicy -c AES-256-CTS -n AES-256-CTS" $dir |& _filter_scratch -$XFS_IO_PROG -c "set_encpolicy -c AES-256-CTS -n AES-256-XTS" $dir |& _filter_scratch -$XFS_IO_PROG -c "set_encpolicy -c AES-256-XTS -n AES-256-XTS" $dir |& _filter_scratch +_set_encpolicy $dir -c AES-256-CTS -n AES-256-CTS |& _filter_scratch +_set_encpolicy $dir -c AES-256-CTS -n AES-256-XTS |& _filter_scratch +_set_encpolicy $dir -c AES-256-XTS -n AES-256-XTS |& _filter_scratch # success, all done status=0 diff --git a/tests/generic/397 b/tests/generic/397 index c2f779a4..a97e866b 100755 --- a/tests/generic/397 +++ b/tests/generic/397 @@ -37,7 +37,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$KEYCTL_PROG" keyctl _new_session_keyring @@ -47,7 +46,7 @@ _scratch_mount mkdir $SCRATCH_MNT/edir $SCRATCH_MNT/ref_dir keydesc=$(_generate_encryption_key) -$XFS_IO_PROG -c "set_encpolicy $keydesc" $SCRATCH_MNT/edir +_set_encpolicy $SCRATCH_MNT/edir $keydesc for dir in $SCRATCH_MNT/edir $SCRATCH_MNT/ref_dir; do touch $dir/empty > /dev/null $XFS_IO_PROG -t -f -c "pwrite 0 4k" $dir/a > /dev/null diff --git a/tests/generic/398 b/tests/generic/398 index ecf82690..b1af65e5 100755 --- a/tests/generic/398 +++ b/tests/generic/398 @@ -56,7 +56,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _requires_renameat2 _new_session_keyring @@ -71,8 +70,8 @@ udir=$SCRATCH_MNT/udir mkdir $edir1 $edir2 $udir keydesc1=$(_generate_encryption_key) keydesc2=$(_generate_encryption_key) -$XFS_IO_PROG -c "set_encpolicy $keydesc1" $edir1 -$XFS_IO_PROG -c "set_encpolicy $keydesc2" $edir2 +_set_encpolicy $edir1 $keydesc1 +_set_encpolicy $edir2 $keydesc2 touch $edir1/efile1 touch $edir2/efile2 touch $udir/ufile diff --git a/tests/generic/399 b/tests/generic/399 index 5e55096f..5625503b 100755 --- a/tests/generic/399 +++ b/tests/generic/399 @@ -43,7 +43,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$XZ_PROG" xz _require_command "$KEYCTL_PROG" keyctl @@ -64,7 +63,7 @@ _scratch_mount keydesc=$(_generate_encryption_key) mkdir $SCRATCH_MNT/encrypted_dir -$XFS_IO_PROG -c "set_encpolicy $keydesc" $SCRATCH_MNT/encrypted_dir +_set_encpolicy $SCRATCH_MNT/encrypted_dir $keydesc # Create the "same" symlink in two different directories. # Later we'll check both the name and target of the symlink. diff --git a/tests/generic/419 b/tests/generic/419 index 1014764c..2f1d34c6 100755 --- a/tests/generic/419 +++ b/tests/generic/419 @@ -38,7 +38,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$KEYCTL_PROG" keyctl _requires_renameat2 @@ -49,7 +48,7 @@ _scratch_mount mkdir $SCRATCH_MNT/edir keydesc=$(_generate_encryption_key) -$XFS_IO_PROG -c "set_encpolicy $keydesc" $SCRATCH_MNT/edir +_set_encpolicy $SCRATCH_MNT/edir $keydesc echo a > $SCRATCH_MNT/edir/a echo b > $SCRATCH_MNT/edir/b _unlink_encryption_key $keydesc diff --git a/tests/generic/421 b/tests/generic/421 index 415e14b5..c8cc2dcc 100755 --- a/tests/generic/421 +++ b/tests/generic/421 @@ -35,7 +35,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$KEYCTL_PROG" keyctl _new_session_keyring @@ -53,7 +52,7 @@ slice=2 rm -rf $dir mkdir $dir keydesc=$(_generate_encryption_key) -$XFS_IO_PROG -c "set_encpolicy $keydesc" $dir +_set_encpolicy $dir $keydesc $XFS_IO_PROG -f $file -c "pwrite 0 $((nproc*slice))M" -c "fsync" > /dev/null # Create processes to read from the encrypted file. Use fadvise to wipe the diff --git a/tests/generic/429 b/tests/generic/429 index 5f5b056a..472fdbd9 100755 --- a/tests/generic/429 +++ b/tests/generic/429 @@ -46,7 +46,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$KEYCTL_PROG" keyctl _require_test_program "t_encrypted_d_revalidate" @@ -58,7 +57,7 @@ keydesc=$(_generate_key_descriptor) raw_key=$(_generate_raw_encryption_key) mkdir $SCRATCH_MNT/edir _add_encryption_key $keydesc $raw_key -$XFS_IO_PROG -c "set_encpolicy $keydesc" $SCRATCH_MNT/edir +_set_encpolicy $SCRATCH_MNT/edir $keydesc # Create two files in the directory: one whose name is valid in the base64 # format used for encoding ciphertext filenames, and one whose name is not. The diff --git a/tests/generic/435 b/tests/generic/435 index baed1bdc..073596f3 100755 --- a/tests/generic/435 +++ b/tests/generic/435 @@ -42,7 +42,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$KEYCTL_PROG" keyctl # set up an encrypted directory @@ -53,7 +52,7 @@ _scratch_mount mkdir $SCRATCH_MNT/edir keydesc=$(_generate_encryption_key) # -f 0x2: zero-pad to 16-byte boundary (i.e. encryption block boundary) -$XFS_IO_PROG -c "set_encpolicy -f 0x2 $keydesc" $SCRATCH_MNT/edir +_set_encpolicy $SCRATCH_MNT/edir $keydesc -f 0x2 # Create files with long names (> 32 bytes, long enough to trigger the use of # "digested" names) in the encrypted directory. diff --git a/tests/generic/440 b/tests/generic/440 index 5013f276..434286f4 100755 --- a/tests/generic/440 +++ b/tests/generic/440 @@ -38,7 +38,6 @@ rm -f $seqres.full _supported_fs generic _supported_os Linux _require_scratch_encryption -_require_xfs_io_command "set_encpolicy" _require_command "$KEYCTL_PROG" keyctl # Set up an encryption-capable filesystem and an encryption key. @@ -52,7 +51,7 @@ _add_encryption_key $keydesc $raw_key # Set up an encrypted directory containing a regular file, a subdirectory, and a # symlink. mkdir $SCRATCH_MNT/edir -$XFS_IO_PROG -c "set_encpolicy $keydesc" $SCRATCH_MNT/edir +_set_encpolicy $SCRATCH_MNT/edir $keydesc mkdir $SCRATCH_MNT/edir/subdir ln -s target $SCRATCH_MNT/edir/symlink echo contents > $SCRATCH_MNT/edir/file @@ -82,7 +81,7 @@ echo echo "***** Child has key, but parent doesn't *****" _add_encryption_key $keydesc $raw_key mkdir $SCRATCH_MNT/edir2 -$XFS_IO_PROG -c "set_encpolicy $keydesc" $SCRATCH_MNT/edir2 +_set_encpolicy $SCRATCH_MNT/edir2 $keydesc ln $SCRATCH_MNT/edir/file $SCRATCH_MNT/edir2/link _scratch_cycle_mount cat $SCRATCH_MNT/edir2/link -- 2.21.0.593.g511ec345e18-goog ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [RFC PATCH 1/7] common/encrypt: introduce helpers for set_encpolicy and get_encpolicy 2019-04-26 20:41 ` [RFC PATCH 1/7] common/encrypt: introduce helpers for set_encpolicy and get_encpolicy Eric Biggers @ 2019-05-12 12:21 ` Eryu Guan 0 siblings, 0 replies; 14+ messages in thread From: Eryu Guan @ 2019-05-12 12:21 UTC (permalink / raw) To: Eric Biggers; +Cc: linux-fscrypt, linux-ext4, fstests, linux-f2fs-devel On Fri, Apr 26, 2019 at 01:41:47PM -0700, Eric Biggers wrote: > From: Eric Biggers <ebiggers@google.com> > > For conciseness in tests, add helper functions that wrap the xfs_io > commands 'set_encpolicy' and 'get_encpolicy'. Then update all > encryption tests to use them. > > Signed-off-by: Eric Biggers <ebiggers@google.com> > --- > common/encrypt | 34 ++++++++++++++++++++++++++++++++-- > tests/ext4/024 | 3 +-- > tests/generic/395 | 28 +++++++++++++--------------- > tests/generic/395.out | 2 +- > tests/generic/396 | 15 +++++++-------- > tests/generic/397 | 3 +-- > tests/generic/398 | 5 ++--- > tests/generic/399 | 3 +-- > tests/generic/419 | 3 +-- > tests/generic/421 | 3 +-- > tests/generic/429 | 3 +-- > tests/generic/435 | 3 +-- > tests/generic/440 | 5 ++--- > 13 files changed, 64 insertions(+), 46 deletions(-) > > diff --git a/common/encrypt b/common/encrypt > index 1b10aa71..54d873fa 100644 > --- a/common/encrypt > +++ b/common/encrypt > @@ -38,8 +38,7 @@ _require_scratch_encryption() > # presence of /sys/fs/ext4/features/encryption, but this is broken on > # some older kernels and is ext4-specific anyway.) > mkdir $SCRATCH_MNT/tmpdir > - if $XFS_IO_PROG -c set_encpolicy $SCRATCH_MNT/tmpdir \ > - 2>&1 >>$seqres.full | \ > + if _set_encpolicy $SCRATCH_MNT/tmpdir 2>&1 >>$seqres.full | \ > egrep -q 'Inappropriate ioctl for device|Operation not supported' > then > _notrun "kernel does not support $FSTYP encryption" > @@ -175,3 +174,34 @@ _revoke_encryption_key() > local keyid=$($KEYCTL_PROG search @s logon $FSTYP:$keydesc) > $KEYCTL_PROG revoke $keyid >>$seqres.full > } > + > +# Set an encryption policy on the specified directory. > +_set_encpolicy() > +{ > + local dir=$1 > + shift > + > + $XFS_IO_PROG -c "set_encpolicy $*" "$dir" > +} > + > +_user_do_set_encpolicy() > +{ > + local dir=$1 > + shift > + > + _user_do "$XFS_IO_PROG -c \"set_encpolicy $*\" \"$dir\"" > +} > + > +_require_get_encpolicy() > +{ > + _require_xfs_io_command "get_encpolicy" > +} This doesn't seem necessary to me, just calling _require_xfs_io_command "get_encpolicy" explicitly is fine to me. Thanks, Eryu ^ permalink raw reply [flat|nested] 14+ messages in thread
* [RFC PATCH 2/7] fscrypt-crypt-util: add utility for reproducing fscrypt encrypted data 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 1/7] common/encrypt: introduce helpers for set_encpolicy and get_encpolicy Eric Biggers @ 2019-04-26 20:41 ` Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 3/7] common/encrypt: support requiring other encryption settings Eric Biggers ` (5 subsequent siblings) 7 siblings, 0 replies; 14+ messages in thread From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel From: Eric Biggers <ebiggers@google.com> Add a utility program that can reproduce encrypted contents and filenames. It implements all encryption algorithms currently supported by fscrypt (a.k.a. ext4, f2fs, and ubifs encryption), and it generates IVs in the same way. The program takes the algorithm and master key on the command line, and encrypts stdin to stdout. A file nonce may also be passed on the command line, and the program will "tweak" the encryption using this nonce in the same way the kernel does -- either by deriving a subkey, or by including the nonce in the IVs. The block size and padding amount may also be specified. No dependencies are added, as all algorithms implemented from scratch. Signed-off-by: Eric Biggers <ebiggers@google.com> --- .gitignore | 1 + src/Makefile | 3 +- src/fscrypt-crypt-util.c | 1645 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 1648 insertions(+), 1 deletion(-) create mode 100644 src/fscrypt-crypt-util.c diff --git a/.gitignore b/.gitignore index c13fb713..11232be7 100644 --- a/.gitignore +++ b/.gitignore @@ -73,6 +73,7 @@ /src/fill /src/fill2 /src/fs_perms +/src/fscrypt-crypt-util /src/fssum /src/fstest /src/fsync-err diff --git a/src/Makefile b/src/Makefile index 4057d7b3..9d3d2529 100644 --- a/src/Makefile +++ b/src/Makefile @@ -27,7 +27,8 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ renameat2 t_getcwd e4compact test-nextquota punch-alternating \ attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \ dio-invalidate-cache stat_test t_encrypted_d_revalidate \ - attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles + attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \ + fscrypt-crypt-util SUBDIRS = log-writes perf diff --git a/src/fscrypt-crypt-util.c b/src/fscrypt-crypt-util.c new file mode 100644 index 00000000..715e2bf2 --- /dev/null +++ b/src/fscrypt-crypt-util.c @@ -0,0 +1,1645 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * fscrypt-crypt-util.c - utility for verifying fscrypt-encrypted data + * + * Copyright 2019 Google LLC + */ + +/* + * This program implements all crypto algorithms supported by fscrypt (a.k.a. + * ext4, f2fs, and ubifs encryption), for the purpose of verifying the + * correctness of the ciphertext stored on-disk. See usage() below. + * + * All algorithms are implemented in portable C code to avoid depending on + * libcrypto (OpenSSL), and because some fscrypt-supported algorithms aren't + * available in libcrypto anyway (e.g. Adiantum). For simplicity, all crypto + * code here tries to follow the mathematical definitions directly, without + * optimizing for performance or worrying about following security best + * practices such as mitigating side-channel attacks. So, only use this program + * for testing! + */ + +#include <asm/byteorder.h> +#include <errno.h> +#include <getopt.h> +#include <limits.h> +#include <linux/types.h> +#include <stdarg.h> +#include <stdbool.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define PROGRAM_NAME "fscrypt-crypt-util" + +/* + * Define to enable the tests of the crypto code in this file. If enabled, you + * must link this program with OpenSSL (-lcrypto) v1.1.0 or later, and your + * kernel needs CONFIG_CRYPTO_USER_API_SKCIPHER=y and CONFIG_CRYPTO_ADIANTUM=y. + */ +#undef ENABLE_ALG_TESTS + +#define NUM_ALG_TEST_ITERATIONS 10000 + +static void usage(FILE *fp) +{ + fputs( +"Usage: " PROGRAM_NAME " [OPTION]... CIPHER MASTER_KEY\n" +"\n" +"Utility for verifying fscrypt-encrypted data. This program encrypts\n" +"(or decrypts) the data on stdin using the given CIPHER with the given\n" +"MASTER_KEY (or a key derived from it, if a KDF is specified), and writes the\n" +"resulting ciphertext (or plaintext) to stdout.\n" +"\n" +"CIPHER can be AES-256-XTS, AES-256-CTS-CBC, AES-128-CBC-ESSIV, AES-128-CTS-CBC,\n" +"or Adiantum. MASTER_KEY must be a hex string long enough for the cipher.\n" +"\n" +"WARNING: this program is only meant for testing, not for \"real\" use!\n" +"\n" +"Options:\n" +" --block-size=BLOCK_SIZE Encrypt each BLOCK_SIZE bytes independently.\n" +" Default: 4096 bytes\n" +" --decrypt Decrypt instead of encrypt\n" +" --file-nonce=NONCE File's nonce as a 32-character hex string\n" +" --kdf=KDF Key derivation function to use: AES-128-ECB\n" +" or none. Default: none\n" +" --help Show this help\n" +" --padding=PADDING If last block is partial, zero-pad it to next\n" +" PADDING-byte boundary. Default: BLOCK_SIZE\n" + , fp); +} + +/*----------------------------------------------------------------------------* + * Utilities * + *----------------------------------------------------------------------------*/ + +#define ARRAY_SIZE(A) (sizeof(A) / sizeof((A)[0])) +#define MIN(x, y) ((x) < (y) ? (x) : (y)) +#define MAX(x, y) ((x) > (y) ? (x) : (y)) +#define ROUND_DOWN(x, y) ((x) & ~((y) - 1)) +#define ROUND_UP(x, y) (((x) + (y) - 1) & ~((y) - 1)) +#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) +#define STATIC_ASSERT(e) ((void)sizeof(char[1 - 2*!(e)])) + +typedef __u8 u8; +typedef __u16 u16; +typedef __u32 u32; +typedef __u64 u64; + +#define cpu_to_le32 __cpu_to_le32 +#define cpu_to_be32 __cpu_to_be32 +#define cpu_to_le64 __cpu_to_le64 +#define cpu_to_be64 __cpu_to_be64 +#define le32_to_cpu __le32_to_cpu +#define be32_to_cpu __be32_to_cpu +#define le64_to_cpu __le64_to_cpu +#define be64_to_cpu __be64_to_cpu + +#define DEFINE_UNALIGNED_ACCESS_HELPERS(type, native_type) \ +static inline native_type __attribute__((unused)) \ +get_unaligned_##type(const void *p) \ +{ \ + __##type x; \ + \ + memcpy(&x, p, sizeof(x)); \ + return type##_to_cpu(x); \ +} \ + \ +static inline void __attribute__((unused)) \ +put_unaligned_##type(native_type v, void *p) \ +{ \ + __##type x = cpu_to_##type(v); \ + \ + memcpy(p, &x, sizeof(x)); \ +} + +DEFINE_UNALIGNED_ACCESS_HELPERS(le32, u32) +DEFINE_UNALIGNED_ACCESS_HELPERS(be32, u32) +DEFINE_UNALIGNED_ACCESS_HELPERS(le64, u64) +DEFINE_UNALIGNED_ACCESS_HELPERS(be64, u64) + +static inline bool is_power_of_2(unsigned long v) +{ + return v != 0 && (v & (v - 1)) == 0; +} + +static inline u32 rol32(u32 v, int n) +{ + return (v << n) | (v >> (32 - n)); +} + +static inline u32 ror32(u32 v, int n) +{ + return (v >> n) | (v << (32 - n)); +} + +static inline u64 ror64(u64 v, int n) +{ + return (v >> n) | (v << (64 - n)); +} + +static inline void xor(u8 *res, const u8 *a, const u8 *b, size_t count) +{ + while (count--) + *res++ = *a++ ^ *b++; +} + +static void __attribute__((noreturn, format(printf, 2, 3))) +do_die(int err, const char *format, ...) +{ + va_list va; + + va_start(va, format); + fputs("[" PROGRAM_NAME "] ERROR: ", stderr); + vfprintf(stderr, format, va); + if (err) + fprintf(stderr, ": %s", strerror(errno)); + putc('\n', stderr); + va_end(va); + exit(1); +} + +#define die(format, ...) do_die(0, (format), ##__VA_ARGS__) +#define die_errno(format, ...) do_die(errno, (format), ##__VA_ARGS__) + +static __attribute__((noreturn)) void +assertion_failed(const char *expr, const char *file, int line) +{ + die("Assertion failed: %s at %s:%d", expr, file, line); +} + +#define ASSERT(e) ({ if (!(e)) assertion_failed(#e, __FILE__, __LINE__); }) + +static void *xmalloc(size_t size) +{ + void *p = malloc(size); + + ASSERT(p != NULL); + return p; +} + +static int hexchar2bin(char c) +{ + if (c >= 'a' && c <= 'f') + return 10 + c - 'a'; + if (c >= 'A' && c <= 'F') + return 10 + c - 'A'; + if (c >= '0' && c <= '9') + return c - '0'; + return -1; +} + +static int hex2bin(const char *hex, u8 *bin, int max_bin_size) +{ + size_t len = strlen(hex); + size_t i; + + if (len & 1) + return -1; + len /= 2; + if (len > max_bin_size) + return -1; + + for (i = 0; i < len; i++) { + int high = hexchar2bin(hex[2 * i]); + int low = hexchar2bin(hex[2 * i + 1]); + + if (high < 0 || low < 0) + return -1; + bin[i] = (high << 4) | low; + } + return len; +} + +static size_t xread(int fd, void *buf, size_t count) +{ + const size_t orig_count = count; + + while (count) { + ssize_t res = read(fd, buf, count); + + if (res < 0) + die_errno("read error"); + if (res == 0) + break; + buf += res; + count -= res; + } + return orig_count - count; +} + +static void full_write(int fd, const void *buf, size_t count) +{ + while (count) { + ssize_t res = write(fd, buf, count); + + if (res < 0) + die_errno("write error"); + buf += res; + count -= res; + } +} + +#ifdef ENABLE_ALG_TESTS +static void rand_bytes(u8 *buf, size_t count) +{ + while (count--) + *buf++ = rand(); +} +#endif + +/*----------------------------------------------------------------------------* + * Finite field arithmetic * + *----------------------------------------------------------------------------*/ + +/* Multiply a GF(2^8) element by the polynomial 'x' */ +static inline u8 gf2_8_mul_x(u8 b) +{ + return (b << 1) ^ ((b & 0x80) ? 0x1B : 0); +} + +/* Multiply four packed GF(2^8) elements by the polynomial 'x' */ +static inline u32 gf2_8_mul_x_4way(u32 w) +{ + return ((w & 0x7F7F7F7F) << 1) ^ (((w & 0x80808080) >> 7) * 0x1B); +} + +/* Element of GF(2^128) */ +typedef struct { + __le64 lo; + __le64 hi; +} ble128; + +/* Multiply a GF(2^128) element by the polynomial 'x' */ +static inline void gf2_128_mul_x(ble128 *t) +{ + u64 lo = le64_to_cpu(t->lo); + u64 hi = le64_to_cpu(t->hi); + + t->hi = cpu_to_le64((hi << 1) | (lo >> 63)); + t->lo = cpu_to_le64((lo << 1) ^ ((hi & (1ULL << 63)) ? 0x87 : 0)); +} + +/*----------------------------------------------------------------------------* + * Group arithmetic * + *----------------------------------------------------------------------------*/ + +/* Element of Z/(2^{128}Z) (a.k.a. the integers modulo 2^128) */ +typedef struct { + __le64 lo; + __le64 hi; +} le128; + +static inline void le128_add(le128 *res, const le128 *a, const le128 *b) +{ + u64 a_lo = le64_to_cpu(a->lo); + u64 b_lo = le64_to_cpu(b->lo); + + res->lo = cpu_to_le64(a_lo + b_lo); + res->hi = cpu_to_le64(le64_to_cpu(a->hi) + le64_to_cpu(b->hi) + + (a_lo + b_lo < a_lo)); +} + +static inline void le128_sub(le128 *res, const le128 *a, const le128 *b) +{ + u64 a_lo = le64_to_cpu(a->lo); + u64 b_lo = le64_to_cpu(b->lo); + + res->lo = cpu_to_le64(a_lo - b_lo); + res->hi = cpu_to_le64(le64_to_cpu(a->hi) - le64_to_cpu(b->hi) - + (a_lo - b_lo > a_lo)); +} + +/*----------------------------------------------------------------------------* + * AES block cipher * + *----------------------------------------------------------------------------*/ + +/* + * Reference: "FIPS 197, Advanced Encryption Standard" + * https://nvlpubs.nist.gov/nistpubs/fips/nist.fips.197.pdf + */ + +#define AES_BLOCK_SIZE 16 +#define AES_128_KEY_SIZE 16 +#define AES_192_KEY_SIZE 24 +#define AES_256_KEY_SIZE 32 + +static inline void AddRoundKey(u32 state[4], const u32 *rk) +{ + int i; + + for (i = 0; i < 4; i++) + state[i] ^= rk[i]; +} + +static const u8 aes_sbox[256] = { + 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, 0x30, 0x01, 0x67, 0x2b, + 0xfe, 0xd7, 0xab, 0x76, 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, + 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, 0xb7, 0xfd, 0x93, 0x26, + 0x36, 0x3f, 0xf7, 0xcc, 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15, + 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, 0x07, 0x12, 0x80, 0xe2, + 0xeb, 0x27, 0xb2, 0x75, 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, + 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, 0x53, 0xd1, 0x00, 0xed, + 0x20, 0xfc, 0xb1, 0x5b, 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf, + 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, 0x45, 0xf9, 0x02, 0x7f, + 0x50, 0x3c, 0x9f, 0xa8, 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, + 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, 0xcd, 0x0c, 0x13, 0xec, + 0x5f, 0x97, 0x44, 0x17, 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73, + 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, 0x46, 0xee, 0xb8, 0x14, + 0xde, 0x5e, 0x0b, 0xdb, 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, + 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, 0xe7, 0xc8, 0x37, 0x6d, + 0x8d, 0xd5, 0x4e, 0xa9, 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08, + 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, 0xe8, 0xdd, 0x74, 0x1f, + 0x4b, 0xbd, 0x8b, 0x8a, 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, + 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, 0xe1, 0xf8, 0x98, 0x11, + 0x69, 0xd9, 0x8e, 0x94, 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf, + 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, 0x41, 0x99, 0x2d, 0x0f, + 0xb0, 0x54, 0xbb, 0x16, +}; + +static u8 aes_inverse_sbox[256]; + +static void aes_init(void) +{ + int i; + + for (i = 0; i < 256; i++) + aes_inverse_sbox[aes_sbox[i]] = i; +} + +static inline u32 DoSubWord(u32 w, const u8 sbox[256]) +{ + return ((u32)sbox[(u8)(w >> 24)] << 24) | + ((u32)sbox[(u8)(w >> 16)] << 16) | + ((u32)sbox[(u8)(w >> 8)] << 8) | + ((u32)sbox[(u8)(w >> 0)] << 0); +} + +static inline u32 SubWord(u32 w) +{ + return DoSubWord(w, aes_sbox); +} + +static inline u32 InvSubWord(u32 w) +{ + return DoSubWord(w, aes_inverse_sbox); +} + +static inline void SubBytes(u32 state[4]) +{ + int i; + + for (i = 0; i < 4; i++) + state[i] = SubWord(state[i]); +} + +static inline void InvSubBytes(u32 state[4]) +{ + int i; + + for (i = 0; i < 4; i++) + state[i] = InvSubWord(state[i]); +} + +static inline void DoShiftRows(u32 state[4], int direction) +{ + u32 newstate[4]; + int i; + + for (i = 0; i < 4; i++) + newstate[i] = (state[(i + direction*0) & 3] & 0xff) | + (state[(i + direction*1) & 3] & 0xff00) | + (state[(i + direction*2) & 3] & 0xff0000) | + (state[(i + direction*3) & 3] & 0xff000000); + memcpy(state, newstate, 16); +} + +static inline void ShiftRows(u32 state[4]) +{ + DoShiftRows(state, 1); +} + +static inline void InvShiftRows(u32 state[4]) +{ + DoShiftRows(state, -1); +} + +/* + * Mix one column by doing the following matrix multiplication in GF(2^8): + * + * | 2 3 1 1 | | w[0] | + * | 1 2 3 1 | | w[1] | + * | 1 1 2 3 | x | w[2] | + * | 3 1 1 2 | | w[3] | + * + * a.k.a. w[i] = 2*w[i] + 3*w[(i+1)%4] + w[(i+2)%4] + w[(i+3)%4] + */ +static inline u32 MixColumn(u32 w) +{ + u32 _2w0_w2 = gf2_8_mul_x_4way(w) ^ ror32(w, 16); + u32 _3w1_w3 = ror32(_2w0_w2 ^ w, 8); + + return _2w0_w2 ^ _3w1_w3; +} + +/* + * ( | 5 0 4 0 | | w[0] | ) + * ( | 0 5 0 4 | | w[1] | ) + * MixColumn( | 4 0 5 0 | x | w[2] | ) + * ( | 0 4 0 5 | | w[3] | ) + */ +static inline u32 InvMixColumn(u32 w) +{ + u32 _4w = gf2_8_mul_x_4way(gf2_8_mul_x_4way(w)); + + return MixColumn(_4w ^ w ^ ror32(_4w, 16)); +} + +static inline void MixColumns(u32 state[4]) +{ + int i; + + for (i = 0; i < 4; i++) + state[i] = MixColumn(state[i]); +} + +static inline void InvMixColumns(u32 state[4]) +{ + int i; + + for (i = 0; i < 4; i++) + state[i] = InvMixColumn(state[i]); +} + +struct aes_key { + u32 round_keys[15 * 4]; + int nrounds; +}; + +/* Expand an AES key */ +static void aes_setkey(struct aes_key *k, const u8 *key, int keysize) +{ + const int N = keysize / 4; + u32 * const rk = k->round_keys; + u8 rcon = 1; + int i; + + ASSERT(keysize == 16 || keysize == 24 || keysize == 32); + k->nrounds = 6 + N; + for (i = 0; i < 4 * (k->nrounds + 1); i++) { + if (i < N) { + rk[i] = get_unaligned_le32(&key[i * sizeof(__le32)]); + } else if (i % N == 0) { + rk[i] = rk[i - N] ^ SubWord(ror32(rk[i - 1], 8)) ^ rcon; + rcon = gf2_8_mul_x(rcon); + } else if (N > 6 && i % N == 4) { + rk[i] = rk[i - N] ^ SubWord(rk[i - 1]); + } else { + rk[i] = rk[i - N] ^ rk[i - 1]; + } + } +} + +/* Encrypt one 16-byte block with AES */ +static void aes_encrypt(const struct aes_key *k, const u8 src[AES_BLOCK_SIZE], + u8 dst[AES_BLOCK_SIZE]) +{ + u32 state[4]; + int i; + + for (i = 0; i < 4; i++) + state[i] = get_unaligned_le32(&src[i * sizeof(__le32)]); + + AddRoundKey(state, k->round_keys); + for (i = 1; i < k->nrounds; i++) { + SubBytes(state); + ShiftRows(state); + MixColumns(state); + AddRoundKey(state, &k->round_keys[4 * i]); + } + SubBytes(state); + ShiftRows(state); + AddRoundKey(state, &k->round_keys[4 * i]); + + for (i = 0; i < 4; i++) + put_unaligned_le32(state[i], &dst[i * sizeof(__le32)]); +} + +/* Decrypt one 16-byte block with AES */ +static void aes_decrypt(const struct aes_key *k, const u8 src[AES_BLOCK_SIZE], + u8 dst[AES_BLOCK_SIZE]) +{ + u32 state[4]; + int i; + + for (i = 0; i < 4; i++) + state[i] = get_unaligned_le32(&src[i * sizeof(__le32)]); + + AddRoundKey(state, &k->round_keys[4 * k->nrounds]); + InvShiftRows(state); + InvSubBytes(state); + for (i = k->nrounds - 1; i >= 1; i--) { + AddRoundKey(state, &k->round_keys[4 * i]); + InvMixColumns(state); + InvShiftRows(state); + InvSubBytes(state); + } + AddRoundKey(state, k->round_keys); + + for (i = 0; i < 4; i++) + put_unaligned_le32(state[i], &dst[i * sizeof(__le32)]); +} + +#ifdef ENABLE_ALG_TESTS +#include <openssl/aes.h> +static void test_aes_keysize(int keysize) +{ + unsigned long num_tests = NUM_ALG_TEST_ITERATIONS; + + while (num_tests--) { + struct aes_key k; + AES_KEY ref_k; + u8 key[AES_256_KEY_SIZE]; + u8 ptext[AES_BLOCK_SIZE]; + u8 ctext[AES_BLOCK_SIZE]; + u8 ref_ctext[AES_BLOCK_SIZE]; + u8 decrypted[AES_BLOCK_SIZE]; + + rand_bytes(key, keysize); + rand_bytes(ptext, AES_BLOCK_SIZE); + + aes_setkey(&k, key, keysize); + aes_encrypt(&k, ptext, ctext); + + ASSERT(AES_set_encrypt_key(key, keysize*8, &ref_k) == 0); + AES_encrypt(ptext, ref_ctext, &ref_k); + + ASSERT(memcmp(ctext, ref_ctext, AES_BLOCK_SIZE) == 0); + + aes_decrypt(&k, ctext, decrypted); + ASSERT(memcmp(ptext, decrypted, AES_BLOCK_SIZE) == 0); + } +} + +static void test_aes(void) +{ + test_aes_keysize(AES_128_KEY_SIZE); + test_aes_keysize(AES_192_KEY_SIZE); + test_aes_keysize(AES_256_KEY_SIZE); +} +#endif /* ENABLE_ALG_TESTS */ + +/*----------------------------------------------------------------------------* + * SHA-256 * + *----------------------------------------------------------------------------*/ + +/* + * Reference: "FIPS 180-2, Secure Hash Standard" + * https://csrc.nist.gov/csrc/media/publications/fips/180/2/archive/2002-08-01/documents/fips180-2withchangenotice.pdf + */ + +#define SHA256_DIGEST_SIZE 32 +#define SHA256_BLOCK_SIZE 64 + +#define Ch(x, y, z) (((x) & (y)) ^ (~(x) & (z))) +#define Maj(x, y, z) (((x) & (y)) ^ ((x) & (z)) ^ ((y) & (z))) +#define Sigma256_0(x) (ror32((x), 2) ^ ror32((x), 13) ^ ror32((x), 22)) +#define Sigma256_1(x) (ror32((x), 6) ^ ror32((x), 11) ^ ror32((x), 25)) +#define sigma256_0(x) (ror32((x), 7) ^ ror32((x), 18) ^ ((x) >> 3)) +#define sigma256_1(x) (ror32((x), 17) ^ ror32((x), 19) ^ ((x) >> 10)) + +static const u32 sha256_iv[8] = { + 0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, 0x510e527f, 0x9b05688c, + 0x1f83d9ab, 0x5be0cd19, +}; + +static const u32 sha256_round_constants[64] = { + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, + 0x923f82a4, 0xab1c5ed5, 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, + 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, 0xe49b69c1, 0xefbe4786, + 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, + 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, + 0x06ca6351, 0x14292967, 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, + 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, 0xa2bfe8a1, 0xa81a664b, + 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, + 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, + 0x5b9cca4f, 0x682e6ff3, 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, + 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, +}; + +/* Compute the SHA-256 digest of the given buffer */ +static void sha256(const u8 *in, size_t inlen, u8 out[SHA256_DIGEST_SIZE]) +{ + const size_t msglen = ROUND_UP(inlen + 9, SHA256_BLOCK_SIZE); + u8 * const msg = xmalloc(msglen); + u32 H[8]; + int i; + + /* super naive way of handling the padding */ + memcpy(msg, in, inlen); + memset(&msg[inlen], 0, msglen - inlen); + msg[inlen] = 0x80; + put_unaligned_be64((u64)inlen * 8, &msg[msglen - sizeof(__be64)]); + in = msg; + + memcpy(H, sha256_iv, sizeof(H)); + do { + u32 a = H[0], b = H[1], c = H[2], d = H[3], + e = H[4], f = H[5], g = H[6], h = H[7]; + u32 W[64]; + + for (i = 0; i < 16; i++) + W[i] = get_unaligned_be32(&in[i * sizeof(__be32)]); + for (; i < ARRAY_SIZE(W); i++) + W[i] = sigma256_1(W[i - 2]) + W[i - 7] + + sigma256_0(W[i - 15]) + W[i - 16]; + for (i = 0; i < ARRAY_SIZE(W); i++) { + u32 T1 = h + Sigma256_1(e) + Ch(e, f, g) + + sha256_round_constants[i] + W[i]; + u32 T2 = Sigma256_0(a) + Maj(a, b, c); + + h = g; g = f; f = e; e = d + T1; + d = c; c = b; b = a; a = T1 + T2; + } + H[0] += a; H[1] += b; H[2] += c; H[3] += d; + H[4] += e; H[5] += f; H[6] += g; H[7] += h; + } while ((in += SHA256_BLOCK_SIZE) != &msg[msglen]); + + for (i = 0; i < ARRAY_SIZE(H); i++) + put_unaligned_be32(H[i], &out[i * sizeof(__be32)]); + free(msg); +} + +#ifdef ENABLE_ALG_TESTS +#include <openssl/sha.h> +static void test_sha2(void) +{ + unsigned long num_tests = NUM_ALG_TEST_ITERATIONS; + + while (num_tests--) { + u8 in[4096]; + u8 digest[SHA256_DIGEST_SIZE]; + u8 ref_digest[SHA256_DIGEST_SIZE]; + const size_t inlen = rand() % (1 + sizeof(in)); + + rand_bytes(in, inlen); + + sha256(in, inlen, digest); + SHA256(in, inlen, ref_digest); + ASSERT(memcmp(digest, ref_digest, SHA256_DIGEST_SIZE) == 0); + } +} +#endif /* ENABLE_ALG_TESTS */ + +/*----------------------------------------------------------------------------* + * AES encryption modes * + *----------------------------------------------------------------------------*/ + +static void aes_256_xts_crypt(const u8 key[2 * AES_256_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], const u8 *src, + u8 *dst, size_t nbytes, bool decrypting) +{ + struct aes_key tweak_key, cipher_key; + ble128 t; + size_t i; + + ASSERT(nbytes % AES_BLOCK_SIZE == 0); + aes_setkey(&cipher_key, key, AES_256_KEY_SIZE); + aes_setkey(&tweak_key, &key[AES_256_KEY_SIZE], AES_256_KEY_SIZE); + aes_encrypt(&tweak_key, iv, (u8 *)&t); + for (i = 0; i < nbytes; i += AES_BLOCK_SIZE) { + xor(&dst[i], &src[i], (const u8 *)&t, AES_BLOCK_SIZE); + if (decrypting) + aes_decrypt(&cipher_key, &dst[i], &dst[i]); + else + aes_encrypt(&cipher_key, &dst[i], &dst[i]); + xor(&dst[i], &dst[i], (const u8 *)&t, AES_BLOCK_SIZE); + gf2_128_mul_x(&t); + } +} + +static void aes_256_xts_encrypt(const u8 key[2 * AES_256_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], const u8 *src, + u8 *dst, size_t nbytes) +{ + aes_256_xts_crypt(key, iv, src, dst, nbytes, false); +} + +static void aes_256_xts_decrypt(const u8 key[2 * AES_256_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], const u8 *src, + u8 *dst, size_t nbytes) +{ + aes_256_xts_crypt(key, iv, src, dst, nbytes, true); +} + +#ifdef ENABLE_ALG_TESTS +#include <openssl/evp.h> +static void test_aes_256_xts(void) +{ + unsigned long num_tests = NUM_ALG_TEST_ITERATIONS; + EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new(); + + ASSERT(ctx != NULL); + while (num_tests--) { + u8 key[2 * AES_256_KEY_SIZE]; + u8 iv[AES_BLOCK_SIZE]; + u8 ptext[512]; + u8 actual_ctext[sizeof(ptext)]; + u8 expected_ctext[sizeof(ptext)]; + u8 decrypted[sizeof(ptext)]; + const size_t datalen = ROUND_DOWN(rand() % (1 + sizeof(ptext)), + AES_BLOCK_SIZE); + int outl; + + rand_bytes(key, sizeof(key)); + rand_bytes(iv, sizeof(iv)); + rand_bytes(ptext, datalen); + + aes_256_xts_encrypt(key, iv, ptext, actual_ctext, datalen); + ASSERT(EVP_EncryptInit_ex(ctx, EVP_aes_256_xts(), + NULL, key, iv) > 0); + ASSERT(EVP_EncryptUpdate(ctx, expected_ctext, &outl, + ptext, datalen) > 0); + ASSERT(outl == datalen); + ASSERT(memcmp(actual_ctext, expected_ctext, datalen) == 0); + + aes_256_xts_decrypt(key, iv, actual_ctext, decrypted, datalen); + ASSERT(memcmp(ptext, decrypted, datalen) == 0); + } + EVP_CIPHER_CTX_free(ctx); +} +#endif /* ENABLE_ALG_TESTS */ + +static void aes_cbc_encrypt(const struct aes_key *k, + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + size_t i; + + ASSERT(nbytes % AES_BLOCK_SIZE == 0); + for (i = 0; i < nbytes; i += AES_BLOCK_SIZE) { + xor(&dst[i], &src[i], (i == 0 ? iv : &dst[i - AES_BLOCK_SIZE]), + AES_BLOCK_SIZE); + aes_encrypt(k, &dst[i], &dst[i]); + } +} + +static void aes_cbc_decrypt(const struct aes_key *k, + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + size_t i = nbytes; + + ASSERT(i % AES_BLOCK_SIZE == 0); + while (i) { + i -= AES_BLOCK_SIZE; + aes_decrypt(k, &src[i], &dst[i]); + xor(&dst[i], &dst[i], (i == 0 ? iv : &src[i - AES_BLOCK_SIZE]), + AES_BLOCK_SIZE); + } +} + +static void aes_cts_cbc_encrypt(const u8 *key, int keysize, + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + const size_t offset = ROUND_DOWN(nbytes - 1, AES_BLOCK_SIZE); + const size_t final_bsize = nbytes - offset; + struct aes_key k; + u8 *pad; + u8 buf[AES_BLOCK_SIZE]; + + ASSERT(nbytes >= AES_BLOCK_SIZE); + + aes_setkey(&k, key, keysize); + + if (nbytes == AES_BLOCK_SIZE) + return aes_cbc_encrypt(&k, iv, src, dst, nbytes); + + aes_cbc_encrypt(&k, iv, src, dst, offset); + pad = &dst[offset - AES_BLOCK_SIZE]; + + memcpy(buf, pad, AES_BLOCK_SIZE); + xor(buf, buf, &src[offset], final_bsize); + memcpy(&dst[offset], pad, final_bsize); + aes_encrypt(&k, buf, pad); +} + +static void aes_cts_cbc_decrypt(const u8 *key, int keysize, + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + const size_t offset = ROUND_DOWN(nbytes - 1, AES_BLOCK_SIZE); + const size_t final_bsize = nbytes - offset; + struct aes_key k; + u8 *pad; + + ASSERT(nbytes >= AES_BLOCK_SIZE); + + aes_setkey(&k, key, keysize); + + if (nbytes == AES_BLOCK_SIZE) + return aes_cbc_decrypt(&k, iv, src, dst, nbytes); + + pad = &dst[offset - AES_BLOCK_SIZE]; + aes_decrypt(&k, &src[offset - AES_BLOCK_SIZE], pad); + xor(&dst[offset], &src[offset], pad, final_bsize); + xor(pad, pad, &dst[offset], final_bsize); + + aes_cbc_decrypt(&k, (offset == AES_BLOCK_SIZE ? + iv : &src[offset - 2 * AES_BLOCK_SIZE]), + pad, pad, AES_BLOCK_SIZE); + aes_cbc_decrypt(&k, iv, src, dst, offset - AES_BLOCK_SIZE); +} + +static void aes_256_cts_cbc_encrypt(const u8 key[AES_256_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + aes_cts_cbc_encrypt(key, AES_256_KEY_SIZE, iv, src, dst, nbytes); +} + +static void aes_256_cts_cbc_decrypt(const u8 key[AES_256_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + aes_cts_cbc_decrypt(key, AES_256_KEY_SIZE, iv, src, dst, nbytes); +} + +#ifdef ENABLE_ALG_TESTS +#include <openssl/modes.h> +static void aes_block128_f(const unsigned char in[16], + unsigned char out[16], const void *key) +{ + aes_encrypt(key, in, out); +} + +static void test_aes_256_cts_cbc(void) +{ + unsigned long num_tests = NUM_ALG_TEST_ITERATIONS; + + while (num_tests--) { + u8 key[AES_256_KEY_SIZE]; + u8 iv[AES_BLOCK_SIZE]; + u8 iv_copy[AES_BLOCK_SIZE]; + u8 ptext[512]; + u8 actual_ctext[sizeof(ptext)]; + u8 expected_ctext[sizeof(ptext)]; + u8 decrypted[sizeof(ptext)]; + const size_t datalen = 16 + (rand() % (sizeof(ptext) - 15)); + struct aes_key k; + + rand_bytes(key, sizeof(key)); + rand_bytes(iv, sizeof(iv)); + rand_bytes(ptext, datalen); + + aes_256_cts_cbc_encrypt(key, iv, ptext, actual_ctext, datalen); + + /* OpenSSL doesn't allow datalen=AES_BLOCK_SIZE; Linux does */ + if (datalen != AES_BLOCK_SIZE) { + aes_setkey(&k, key, sizeof(key)); + memcpy(iv_copy, iv, sizeof(iv)); + ASSERT(CRYPTO_cts128_encrypt_block(ptext, + expected_ctext, + datalen, &k, iv_copy, + aes_block128_f) + == datalen); + ASSERT(memcmp(actual_ctext, expected_ctext, + datalen) == 0); + } + aes_256_cts_cbc_decrypt(key, iv, actual_ctext, decrypted, + datalen); + ASSERT(memcmp(ptext, decrypted, datalen) == 0); + } +} +#endif /* ENABLE_ALG_TESTS */ + +static void essiv_generate_iv(const u8 orig_key[AES_128_KEY_SIZE], + const u8 orig_iv[AES_BLOCK_SIZE], + u8 real_iv[AES_BLOCK_SIZE]) +{ + u8 essiv_key[SHA256_DIGEST_SIZE]; + struct aes_key essiv; + + /* AES encrypt the original IV using a hash of the original key */ + STATIC_ASSERT(SHA256_DIGEST_SIZE == AES_256_KEY_SIZE); + sha256(orig_key, AES_128_KEY_SIZE, essiv_key); + aes_setkey(&essiv, essiv_key, AES_256_KEY_SIZE); + aes_encrypt(&essiv, orig_iv, real_iv); +} + +static void aes_128_cbc_essiv_encrypt(const u8 key[AES_128_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + struct aes_key k; + u8 real_iv[AES_BLOCK_SIZE]; + + aes_setkey(&k, key, AES_128_KEY_SIZE); + essiv_generate_iv(key, iv, real_iv); + aes_cbc_encrypt(&k, real_iv, src, dst, nbytes); +} + +static void aes_128_cbc_essiv_decrypt(const u8 key[AES_128_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + struct aes_key k; + u8 real_iv[AES_BLOCK_SIZE]; + + aes_setkey(&k, key, AES_128_KEY_SIZE); + essiv_generate_iv(key, iv, real_iv); + aes_cbc_decrypt(&k, real_iv, src, dst, nbytes); +} + +static void aes_128_cts_cbc_encrypt(const u8 key[AES_128_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + aes_cts_cbc_encrypt(key, AES_128_KEY_SIZE, iv, src, dst, nbytes); +} + +static void aes_128_cts_cbc_decrypt(const u8 key[AES_128_KEY_SIZE], + const u8 iv[AES_BLOCK_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + aes_cts_cbc_decrypt(key, AES_128_KEY_SIZE, iv, src, dst, nbytes); +} + +/*----------------------------------------------------------------------------* + * XChaCha12 stream cipher * + *----------------------------------------------------------------------------*/ + +/* + * References: + * - "XChaCha: eXtended-nonce ChaCha and AEAD_XChaCha20_Poly1305" + * https://tools.ietf.org/html/draft-arciszewski-xchacha-03 + * + * - "ChaCha, a variant of Salsa20" + * https://cr.yp.to/chacha/chacha-20080128.pdf + * + * - "Extending the Salsa20 nonce" + * https://cr.yp.to/snuffle/xsalsa-20081128.pdf + */ + +#define CHACHA_KEY_SIZE 32 +#define XCHACHA_KEY_SIZE CHACHA_KEY_SIZE +#define XCHACHA_NONCE_SIZE 24 + +static void chacha_init_state(u32 state[16], const u8 key[CHACHA_KEY_SIZE], + const u8 iv[16]) +{ + static const u8 consts[16] = "expand 32-byte k"; + int i; + + for (i = 0; i < 4; i++) + state[i] = get_unaligned_le32(&consts[i * sizeof(__le32)]); + for (i = 0; i < 8; i++) + state[4 + i] = get_unaligned_le32(&key[i * sizeof(__le32)]); + for (i = 0; i < 4; i++) + state[12 + i] = get_unaligned_le32(&iv[i * sizeof(__le32)]); +} + +#define CHACHA_QUARTERROUND(a, b, c, d) \ + do { \ + a += b; d = rol32(d ^ a, 16); \ + c += d; b = rol32(b ^ c, 12); \ + a += b; d = rol32(d ^ a, 8); \ + c += d; b = rol32(b ^ c, 7); \ + } while (0) + +static void chacha_permute(u32 x[16], int nrounds) +{ + do { + /* column round */ + CHACHA_QUARTERROUND(x[0], x[4], x[8], x[12]); + CHACHA_QUARTERROUND(x[1], x[5], x[9], x[13]); + CHACHA_QUARTERROUND(x[2], x[6], x[10], x[14]); + CHACHA_QUARTERROUND(x[3], x[7], x[11], x[15]); + + /* diagonal round */ + CHACHA_QUARTERROUND(x[0], x[5], x[10], x[15]); + CHACHA_QUARTERROUND(x[1], x[6], x[11], x[12]); + CHACHA_QUARTERROUND(x[2], x[7], x[8], x[13]); + CHACHA_QUARTERROUND(x[3], x[4], x[9], x[14]); + } while ((nrounds -= 2) != 0); +} + +static void xchacha(const u8 key[XCHACHA_KEY_SIZE], + const u8 nonce[XCHACHA_NONCE_SIZE], + const u8 *src, u8 *dst, size_t nbytes, int nrounds) +{ + u32 state[16]; + u8 real_key[CHACHA_KEY_SIZE]; + u8 real_iv[16] = { 0 }; + size_t i, j; + + /* Compute real key using original key and first 128 nonce bits */ + chacha_init_state(state, key, nonce); + chacha_permute(state, nrounds); + for (i = 0; i < 8; i++) /* state words 0..3, 12..15 */ + put_unaligned_le32(state[(i < 4 ? 0 : 8) + i], + &real_key[i * sizeof(__le32)]); + + /* Now do regular ChaCha, using real key and remaining nonce bits */ + memcpy(&real_iv[8], nonce + 16, 8); + chacha_init_state(state, real_key, real_iv); + for (i = 0; i < nbytes; i += 64) { + u32 x[16]; + __le32 keystream[16]; + + memcpy(x, state, 64); + chacha_permute(x, nrounds); + for (j = 0; j < 16; j++) + keystream[j] = cpu_to_le32(x[j] + state[j]); + xor(&dst[i], &src[i], (u8 *)keystream, MIN(nbytes - i, 64)); + if (++state[12] == 0) + state[13]++; + } +} + +static void xchacha12(const u8 key[XCHACHA_KEY_SIZE], + const u8 nonce[XCHACHA_NONCE_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + xchacha(key, nonce, src, dst, nbytes, 12); +} + +/*----------------------------------------------------------------------------* + * Poly1305 * + *----------------------------------------------------------------------------*/ + +/* + * Note: this is only the Poly1305 ε-almost-∆-universal hash function, not the + * full Poly1305 MAC. I.e., it doesn't add anything at the end. + */ + +#define POLY1305_KEY_SIZE 16 +#define POLY1305_BLOCK_SIZE 16 + +static void poly1305(const u8 key[POLY1305_KEY_SIZE], + const u8 *msg, size_t msglen, le128 *out) +{ + const u32 limb_mask = 0x3ffffff; /* limbs are base 2^26 */ + const u64 r0 = (get_unaligned_le32(key + 0) >> 0) & 0x3ffffff; + const u64 r1 = (get_unaligned_le32(key + 3) >> 2) & 0x3ffff03; + const u64 r2 = (get_unaligned_le32(key + 6) >> 4) & 0x3ffc0ff; + const u64 r3 = (get_unaligned_le32(key + 9) >> 6) & 0x3f03fff; + const u64 r4 = (get_unaligned_le32(key + 12) >> 8) & 0x00fffff; + u32 h0 = 0, h1 = 0, h2 = 0, h3 = 0, h4 = 0; + u32 g0, g1, g2, g3, g4, ge_p_mask; + + /* Partial block support is not necessary for Adiantum */ + ASSERT(msglen % POLY1305_BLOCK_SIZE == 0); + + while (msglen) { + u64 d0, d1, d2, d3, d4; + + /* h += *msg */ + h0 += (get_unaligned_le32(msg + 0) >> 0) & limb_mask; + h1 += (get_unaligned_le32(msg + 3) >> 2) & limb_mask; + h2 += (get_unaligned_le32(msg + 6) >> 4) & limb_mask; + h3 += (get_unaligned_le32(msg + 9) >> 6) & limb_mask; + h4 += (get_unaligned_le32(msg + 12) >> 8) | (1 << 24); + + /* h *= r */ + d0 = h0*r0 + h1*5*r4 + h2*5*r3 + h3*5*r2 + h4*5*r1; + d1 = h0*r1 + h1*r0 + h2*5*r4 + h3*5*r3 + h4*5*r2; + d2 = h0*r2 + h1*r1 + h2*r0 + h3*5*r4 + h4*5*r3; + d3 = h0*r3 + h1*r2 + h2*r1 + h3*r0 + h4*5*r4; + d4 = h0*r4 + h1*r3 + h2*r2 + h3*r1 + h4*r0; + + /* (partial) h %= 2^130 - 5 */ + d1 += d0 >> 26; h0 = d0 & limb_mask; + d2 += d1 >> 26; h1 = d1 & limb_mask; + d3 += d2 >> 26; h2 = d2 & limb_mask; + d4 += d3 >> 26; h3 = d3 & limb_mask; + h0 += (d4 >> 26) * 5; h4 = d4 & limb_mask; + h1 += h0 >> 26; h0 &= limb_mask; + + msg += POLY1305_BLOCK_SIZE; + msglen -= POLY1305_BLOCK_SIZE; + } + + /* fully carry h */ + h2 += (h1 >> 26); h1 &= limb_mask; + h3 += (h2 >> 26); h2 &= limb_mask; + h4 += (h3 >> 26); h3 &= limb_mask; + h0 += (h4 >> 26) * 5; h4 &= limb_mask; + h1 += (h0 >> 26); h0 &= limb_mask; + + /* if (h >= 2^130 - 5) h -= 2^130 - 5; */ + g0 = h0 + 5; + g1 = h1 + (g0 >> 26); g0 &= limb_mask; + g2 = h2 + (g1 >> 26); g1 &= limb_mask; + g3 = h3 + (g2 >> 26); g2 &= limb_mask; + g4 = h4 + (g3 >> 26); g3 &= limb_mask; + ge_p_mask = ~((g4 >> 26) - 1); /* all 1's if h >= 2^130 - 5, else 0 */ + h0 = (h0 & ~ge_p_mask) | (g0 & ge_p_mask); + h1 = (h1 & ~ge_p_mask) | (g1 & ge_p_mask); + h2 = (h2 & ~ge_p_mask) | (g2 & ge_p_mask); + h3 = (h3 & ~ge_p_mask) | (g3 & ge_p_mask); + h4 = (h4 & ~ge_p_mask) | (g4 & ge_p_mask & limb_mask); + + /* h %= 2^128 */ + out->lo = cpu_to_le64(((u64)h2 << 52) | ((u64)h1 << 26) | h0); + out->hi = cpu_to_le64(((u64)h4 << 40) | ((u64)h3 << 14) | (h2 >> 12)); +} + +/*----------------------------------------------------------------------------* + * Adiantum encryption mode * + *----------------------------------------------------------------------------*/ + +/* + * Reference: "Adiantum: length-preserving encryption for entry-level processors" + * https://tosc.iacr.org/index.php/ToSC/article/view/7360 + */ + +#define ADIANTUM_KEY_SIZE 32 +#define ADIANTUM_IV_SIZE 32 +#define ADIANTUM_HASH_KEY_SIZE ((2 * POLY1305_KEY_SIZE) + NH_KEY_SIZE) + +#define NH_KEY_SIZE 1072 +#define NH_KEY_WORDS (NH_KEY_SIZE / sizeof(u32)) +#define NH_BLOCK_SIZE 1024 +#define NH_HASH_SIZE 32 +#define NH_MESSAGE_UNIT 16 + +static u64 nh_pass(const u32 *key, const u8 *msg, size_t msglen) +{ + u64 sum = 0; + + ASSERT(msglen % NH_MESSAGE_UNIT == 0); + while (msglen) { + sum += (u64)(u32)(get_unaligned_le32(msg + 0) + key[0]) * + (u32)(get_unaligned_le32(msg + 8) + key[2]); + sum += (u64)(u32)(get_unaligned_le32(msg + 4) + key[1]) * + (u32)(get_unaligned_le32(msg + 12) + key[3]); + key += NH_MESSAGE_UNIT / sizeof(key[0]); + msg += NH_MESSAGE_UNIT; + msglen -= NH_MESSAGE_UNIT; + } + return sum; +} + +/* NH ε-almost-universal hash function */ +static void nh(const u32 *key, const u8 *msg, size_t msglen, + u8 result[NH_HASH_SIZE]) +{ + size_t i; + + for (i = 0; i < NH_HASH_SIZE; i += sizeof(__le64)) { + put_unaligned_le64(nh_pass(key, msg, msglen), &result[i]); + key += NH_MESSAGE_UNIT / sizeof(key[0]); + } +} + +/* Adiantum's ε-almost-∆-universal hash function */ +static void adiantum_hash(const u8 key[ADIANTUM_HASH_KEY_SIZE], + const u8 iv[ADIANTUM_IV_SIZE], + const u8 *msg, size_t msglen, le128 *result) +{ + const u8 *header_poly_key = key; + const u8 *msg_poly_key = header_poly_key + POLY1305_KEY_SIZE; + const u8 *nh_key = msg_poly_key + POLY1305_KEY_SIZE; + u32 nh_key_words[NH_KEY_WORDS]; + u8 header[ADIANTUM_IV_SIZE + POLY1305_BLOCK_SIZE]; + const size_t num_nh_blocks = DIV_ROUND_UP(msglen, NH_BLOCK_SIZE); + u8 *nh_hashes = xmalloc(num_nh_blocks * NH_HASH_SIZE); + const size_t padded_msglen = ROUND_UP(msglen, NH_MESSAGE_UNIT); + u8 *padded_msg = xmalloc(padded_msglen); + le128 hash1, hash2; + size_t i; + + for (i = 0; i < NH_KEY_WORDS; i++) + nh_key_words[i] = get_unaligned_le32(&nh_key[i * sizeof(u32)]); + + /* Hash tweak and message length with first Poly1305 key */ + put_unaligned_le64((u64)msglen * 8, header); + put_unaligned_le64(0, &header[sizeof(__le64)]); + memcpy(&header[POLY1305_BLOCK_SIZE], iv, ADIANTUM_IV_SIZE); + poly1305(header_poly_key, header, sizeof(header), &hash1); + + /* Hash NH hashes of message blocks using second Poly1305 key */ + /* (using a super naive way of handling the padding) */ + memcpy(padded_msg, msg, msglen); + memset(&padded_msg[msglen], 0, padded_msglen - msglen); + for (i = 0; i < num_nh_blocks; i++) { + nh(nh_key_words, &padded_msg[i * NH_BLOCK_SIZE], + MIN(NH_BLOCK_SIZE, padded_msglen - (i * NH_BLOCK_SIZE)), + &nh_hashes[i * NH_HASH_SIZE]); + } + poly1305(msg_poly_key, nh_hashes, num_nh_blocks * NH_HASH_SIZE, &hash2); + + /* Add the two hashes together to get the final hash */ + le128_add(result, &hash1, &hash2); + + free(nh_hashes); + free(padded_msg); +} + +static void adiantum_crypt(const u8 key[ADIANTUM_KEY_SIZE], + const u8 iv[ADIANTUM_IV_SIZE], const u8 *src, + u8 *dst, size_t nbytes, bool decrypting) +{ + u8 subkeys[AES_256_KEY_SIZE + ADIANTUM_HASH_KEY_SIZE] = { 0 }; + struct aes_key aes_key; + union { + u8 nonce[XCHACHA_NONCE_SIZE]; + le128 block; + } u = { .nonce = { 1 } }; + const size_t bulk_len = nbytes - sizeof(u.block); + le128 hash; + + ASSERT(nbytes >= sizeof(u.block)); + + /* Derive subkeys */ + xchacha12(key, u.nonce, subkeys, subkeys, sizeof(subkeys)); + aes_setkey(&aes_key, subkeys, AES_256_KEY_SIZE); + + /* Hash left part and add to right part */ + adiantum_hash(&subkeys[AES_256_KEY_SIZE], iv, src, bulk_len, &hash); + memcpy(&u.block, &src[bulk_len], sizeof(u.block)); + le128_add(&u.block, &u.block, &hash); + + if (!decrypting) /* Encrypt right part with block cipher */ + aes_encrypt(&aes_key, u.nonce, u.nonce); + + /* Encrypt left part with stream cipher, using the computed nonce */ + u.nonce[sizeof(u.block)] = 1; + xchacha12(key, u.nonce, src, dst, bulk_len); + + if (decrypting) /* Decrypt right part with block cipher */ + aes_decrypt(&aes_key, u.nonce, u.nonce); + + /* Finalize right part by subtracting hash of left part */ + adiantum_hash(&subkeys[AES_256_KEY_SIZE], iv, dst, bulk_len, &hash); + le128_sub(&u.block, &u.block, &hash); + memcpy(&dst[bulk_len], &u.block, sizeof(u.block)); +} + +static void adiantum_encrypt(const u8 key[ADIANTUM_KEY_SIZE], + const u8 iv[ADIANTUM_IV_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + adiantum_crypt(key, iv, src, dst, nbytes, false); +} + +static void adiantum_decrypt(const u8 key[ADIANTUM_KEY_SIZE], + const u8 iv[ADIANTUM_IV_SIZE], + const u8 *src, u8 *dst, size_t nbytes) +{ + adiantum_crypt(key, iv, src, dst, nbytes, true); +} + +#ifdef ENABLE_ALG_TESTS +#include <linux/if_alg.h> +#include <sys/socket.h> +#define SOL_ALG 279 +static void af_alg_crypt(int algfd, int op, const u8 *key, size_t keylen, + const u8 *iv, size_t ivlen, + const u8 *src, u8 *dst, size_t datalen) +{ + size_t controllen = CMSG_SPACE(sizeof(int)) + + CMSG_SPACE(sizeof(struct af_alg_iv) + ivlen); + u8 *control = xmalloc(controllen); + struct iovec iov = { .iov_base = (u8 *)src, .iov_len = datalen }; + struct msghdr msg = { + .msg_iov = &iov, + .msg_iovlen = 1, + .msg_control = control, + .msg_controllen = controllen, + }; + struct cmsghdr *cmsg; + struct af_alg_iv *algiv; + int reqfd; + + memset(control, 0, controllen); + + cmsg = CMSG_FIRSTHDR(&msg); + cmsg->cmsg_len = CMSG_LEN(sizeof(int)); + cmsg->cmsg_level = SOL_ALG; + cmsg->cmsg_type = ALG_SET_OP; + *(int *)CMSG_DATA(cmsg) = op; + + cmsg = CMSG_NXTHDR(&msg, cmsg); + cmsg->cmsg_len = CMSG_LEN(sizeof(struct af_alg_iv) + ivlen); + cmsg->cmsg_level = SOL_ALG; + cmsg->cmsg_type = ALG_SET_IV; + algiv = (struct af_alg_iv *)CMSG_DATA(cmsg); + algiv->ivlen = ivlen; + memcpy(algiv->iv, iv, ivlen); + + if (setsockopt(algfd, SOL_ALG, ALG_SET_KEY, key, keylen) != 0) + die_errno("can't set key on AF_ALG socket"); + + reqfd = accept(algfd, NULL, NULL); + if (reqfd < 0) + die_errno("can't accept() AF_ALG socket"); + if (sendmsg(reqfd, &msg, 0) != datalen) + die_errno("can't sendmsg() AF_ALG request socket"); + if (xread(reqfd, dst, datalen) != datalen) + die("short read from AF_ALG request socket"); + close(reqfd); + + free(control); +} + +static void test_adiantum(void) +{ + int algfd = socket(AF_ALG, SOCK_SEQPACKET, 0); + struct sockaddr_alg addr = { + .salg_type = "skcipher", + .salg_name = "adiantum(xchacha12,aes)", + }; + unsigned long num_tests = NUM_ALG_TEST_ITERATIONS; + + if (algfd < 0) + die_errno("can't create AF_ALG socket"); + if (bind(algfd, (struct sockaddr *)&addr, sizeof(addr)) != 0) + die_errno("can't bind AF_ALG socket to Adiantum algorithm"); + + while (num_tests--) { + u8 key[ADIANTUM_KEY_SIZE]; + u8 iv[ADIANTUM_IV_SIZE]; + u8 ptext[4096]; + u8 actual_ctext[sizeof(ptext)]; + u8 expected_ctext[sizeof(ptext)]; + u8 decrypted[sizeof(ptext)]; + const size_t datalen = 16 + (rand() % (sizeof(ptext) - 15)); + + rand_bytes(key, sizeof(key)); + rand_bytes(iv, sizeof(iv)); + rand_bytes(ptext, datalen); + + adiantum_encrypt(key, iv, ptext, actual_ctext, datalen); + af_alg_crypt(algfd, ALG_OP_ENCRYPT, key, sizeof(key), + iv, sizeof(iv), ptext, expected_ctext, datalen); + ASSERT(memcmp(actual_ctext, expected_ctext, datalen) == 0); + + adiantum_decrypt(key, iv, actual_ctext, decrypted, datalen); + ASSERT(memcmp(ptext, decrypted, datalen) == 0); + } + close(algfd); +} +#endif /* ENABLE_ALG_TESTS */ + +/*----------------------------------------------------------------------------* + * Main program * + *----------------------------------------------------------------------------*/ + +#define FILE_NONCE_SIZE 16 +#define MAX_KEY_SIZE 64 + +static const struct fscrypt_cipher { + const char *name; + void (*encrypt)(const u8 *key, const u8 *iv, const u8 *src, + u8 *dst, size_t nbytes); + void (*decrypt)(const u8 *key, const u8 *iv, const u8 *src, + u8 *dst, size_t nbytes); + int keysize; + int min_input_size; +} fscrypt_ciphers[] = { + { + .name = "AES-256-XTS", + .encrypt = aes_256_xts_encrypt, + .decrypt = aes_256_xts_decrypt, + .keysize = 2 * AES_256_KEY_SIZE, + .min_input_size = AES_BLOCK_SIZE, + }, { + .name = "AES-256-CTS-CBC", + .encrypt = aes_256_cts_cbc_encrypt, + .decrypt = aes_256_cts_cbc_decrypt, + .keysize = AES_256_KEY_SIZE, + .min_input_size = AES_BLOCK_SIZE, + }, { + .name = "AES-128-CBC-ESSIV", + .encrypt = aes_128_cbc_essiv_encrypt, + .decrypt = aes_128_cbc_essiv_decrypt, + .keysize = AES_128_KEY_SIZE, + .min_input_size = AES_BLOCK_SIZE, + }, { + .name = "AES-128-CTS-CBC", + .encrypt = aes_128_cts_cbc_encrypt, + .decrypt = aes_128_cts_cbc_decrypt, + .keysize = AES_128_KEY_SIZE, + .min_input_size = AES_BLOCK_SIZE, + }, { + .name = "Adiantum", + .encrypt = adiantum_encrypt, + .decrypt = adiantum_decrypt, + .keysize = ADIANTUM_KEY_SIZE, + .min_input_size = AES_BLOCK_SIZE, + } +}; + +static const struct fscrypt_cipher *find_fscrypt_cipher(const char *name) +{ + size_t i; + + for (i = 0; i < ARRAY_SIZE(fscrypt_ciphers); i++) { + if (strcmp(fscrypt_ciphers[i].name, name) == 0) + return &fscrypt_ciphers[i]; + } + return NULL; +} + +struct fscrypt_iv { + union { + __le64 block_num; + u8 bytes[32]; + }; +}; + +static void crypt_loop(const struct fscrypt_cipher *cipher, const u8 *key, + struct fscrypt_iv *iv, bool decrypting, + size_t block_size, size_t padding) +{ + u8 *buf = xmalloc(block_size); + size_t res; + + while ((res = xread(STDIN_FILENO, buf, block_size)) > 0) { + size_t crypt_len = block_size; + + if (padding > 0) + crypt_len = MIN(ROUND_UP(res, padding), crypt_len); + + crypt_len = MAX(crypt_len, cipher->min_input_size); + + memset(&buf[res], 0, crypt_len - res); + + if (decrypting) + cipher->decrypt(key, iv->bytes, buf, buf, crypt_len); + else + cipher->encrypt(key, iv->bytes, buf, buf, crypt_len); + + full_write(STDOUT_FILENO, buf, crypt_len); + + iv->block_num = cpu_to_le64(le64_to_cpu(iv->block_num) + 1); + } + free(buf); +} + +/* The supported key derivation functions */ +enum kdf_algorithm { + KDF_NONE, + KDF_AES_128_ECB, +}; + +static enum kdf_algorithm parse_kdf_algorithm(const char *arg) +{ + if (strcmp(arg, "none") == 0) + return KDF_NONE; + if (strcmp(arg, "AES-128-ECB") == 0) + return KDF_AES_128_ECB; + die("Unknown KDF: %s", arg); +} + +/* + * Get the key and starting IV with which the encryption will actually be done. + * If a KDF was specified, a subkey is derived from the master key and file + * nonce. Otherwise, the master key is used directly. + */ +static void get_key_and_iv(const u8 *master_key, size_t master_key_size, + enum kdf_algorithm kdf, + const u8 nonce[FILE_NONCE_SIZE], + u8 *real_key, size_t real_key_size, + struct fscrypt_iv *iv) +{ + bool nonce_in_iv = false; + struct aes_key aes_key; + size_t i; + + ASSERT(real_key_size <= master_key_size); + + memset(iv, 0, sizeof(*iv)); + + switch (kdf) { + case KDF_NONE: + memcpy(real_key, master_key, real_key_size); + nonce_in_iv = true; + break; + case KDF_AES_128_ECB: + if (nonce == NULL) + die("--file-nonce is required with --kdf=AES-128-ECB"); + STATIC_ASSERT(FILE_NONCE_SIZE == AES_128_KEY_SIZE); + ASSERT(real_key_size % AES_BLOCK_SIZE == 0); + aes_setkey(&aes_key, nonce, AES_128_KEY_SIZE); + for (i = 0; i < real_key_size; i += AES_BLOCK_SIZE) + aes_encrypt(&aes_key, &master_key[i], &real_key[i]); + break; + default: + ASSERT(0); + } + + if (nonce_in_iv && nonce != NULL) + memcpy(&iv->bytes[8], nonce, FILE_NONCE_SIZE); +} + +enum { + OPT_BLOCK_SIZE, + OPT_DECRYPT, + OPT_FILE_NONCE, + OPT_HELP, + OPT_KDF, + OPT_PADDING, +}; + +static const struct option longopts[] = { + { "block-size", required_argument, NULL, OPT_BLOCK_SIZE }, + { "decrypt", no_argument, NULL, OPT_DECRYPT }, + { "file-nonce", required_argument, NULL, OPT_FILE_NONCE }, + { "help", no_argument, NULL, OPT_HELP }, + { "kdf", required_argument, NULL, OPT_KDF }, + { "padding", required_argument, NULL, OPT_PADDING }, + { NULL, 0, NULL, 0 }, +}; + +int main(int argc, char *argv[]) +{ + size_t block_size = 4096; + bool decrypting = false; + u8 _file_nonce[FILE_NONCE_SIZE]; + u8 *file_nonce = NULL; + enum kdf_algorithm kdf = KDF_NONE; + size_t padding = 0; + const struct fscrypt_cipher *cipher; + u8 master_key[MAX_KEY_SIZE]; + int master_key_size; + u8 real_key[MAX_KEY_SIZE]; + struct fscrypt_iv iv; + char *tmp; + int c; + + aes_init(); + +#ifdef ENABLE_ALG_TESTS + test_aes(); + test_sha2(); + test_aes_256_xts(); + test_aes_256_cts_cbc(); + test_adiantum(); +#endif + + while ((c = getopt_long(argc, argv, "", longopts, NULL)) != -1) { + switch (c) { + case OPT_BLOCK_SIZE: + block_size = strtoul(optarg, &tmp, 10); + if (block_size <= 0 || *tmp) + die("Invalid block size: %s", optarg); + break; + case OPT_DECRYPT: + decrypting = true; + break; + case OPT_FILE_NONCE: + if (hex2bin(optarg, _file_nonce, FILE_NONCE_SIZE) != + FILE_NONCE_SIZE) + die("Invalid file nonce: %s", optarg); + file_nonce = _file_nonce; + break; + case OPT_HELP: + usage(stdout); + return 0; + case OPT_KDF: + kdf = parse_kdf_algorithm(optarg); + break; + case OPT_PADDING: + padding = strtoul(optarg, &tmp, 10); + if (padding <= 0 || *tmp || !is_power_of_2(padding) || + padding > INT_MAX) + die("Invalid padding amount: %s", optarg); + break; + default: + usage(stderr); + return 2; + } + } + argc -= optind; + argv += optind; + + if (argc != 2) { + usage(stderr); + return 2; + } + + cipher = find_fscrypt_cipher(argv[0]); + if (cipher == NULL) + die("Unknown cipher: %s", argv[0]); + + if (block_size < cipher->min_input_size) + die("Block size of %zu bytes is too small for cipher %s", + block_size, cipher->name); + + master_key_size = hex2bin(argv[1], master_key, MAX_KEY_SIZE); + if (master_key_size < 0) + die("Invalid master_key: %s", argv[1]); + if (master_key_size < cipher->keysize) + die("Master key is too short for cipher %s", cipher->name); + + get_key_and_iv(master_key, master_key_size, kdf, file_nonce, + real_key, cipher->keysize, &iv); + + crypt_loop(cipher, real_key, &iv, decrypting, block_size, padding); + return 0; +} -- 2.21.0.593.g511ec345e18-goog _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [RFC PATCH 3/7] common/encrypt: support requiring other encryption settings 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 1/7] common/encrypt: introduce helpers for set_encpolicy and get_encpolicy Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 2/7] fscrypt-crypt-util: add utility for reproducing fscrypt encrypted data Eric Biggers @ 2019-04-26 20:41 ` Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests Eric Biggers ` (4 subsequent siblings) 7 siblings, 0 replies; 14+ messages in thread From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel From: Eric Biggers <ebiggers@google.com> Update _require_scratch_encryption() to support checking for kernel support for contents and filenames encryption modes besides the default. This will be used by some of the ciphertext verification tests. Signed-off-by: Eric Biggers <ebiggers@google.com> --- common/encrypt | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/common/encrypt b/common/encrypt index 54d873fa..37f16b94 100644 --- a/common/encrypt +++ b/common/encrypt @@ -4,6 +4,15 @@ # # Functions for setting up and testing file encryption +# +# _require_scratch_encryption [-c CONTENTS_MODE] [-n FILENAMES_MODE] +# +# Require encryption support on the scratch device. +# +# This checks for support for the default type of encryption policy (AES-256-XTS +# and AES-256-CTS). Options can be specified to also require support for a +# different type of encryption policy. +# _require_scratch_encryption() { _require_scratch @@ -44,9 +53,58 @@ _require_scratch_encryption() _notrun "kernel does not support $FSTYP encryption" fi rmdir $SCRATCH_MNT/tmpdir + + # If required, check for support for the specific type of encryption + # policy required by the test. + if [ $# -ne 0 ]; then + _require_encryption_policy_support $SCRATCH_MNT "$@" + fi + _scratch_unmount } +_require_encryption_policy_support() +{ + local mnt=$1 + local dir=$mnt/tmpdir + local set_encpolicy_args="" + local c + + OPTIND=2 + while getopts "c:n:" c; do + case $c in + c|n) + set_encpolicy_args+=" -$c $OPTARG" + ;; + *) + _fail "Unrecognized option '$c'" + ;; + esac + done + set_encpolicy_args=${set_encpolicy_args# } + + echo "Checking whether kernel supports encryption policy: $set_encpolicy_args" \ + >> $seqres.full + + mkdir $dir + _require_command "$KEYCTL_PROG" keyctl + _new_session_keyring + local keydesc=$(_generate_encryption_key) + if _set_encpolicy $dir $keydesc $set_encpolicy_args \ + 2>&1 >>$seqres.full | egrep -q 'Invalid argument'; then + _notrun "kernel does not support encryption policy: '$set_encpolicy_args'" + fi + # fscrypt allows setting policies with modes it knows about, even + # without kernel crypto API support. E.g. a policy using Adiantum + # encryption can be set on a kernel without CONFIG_CRYPTO_ADIANTUM. + # But actually trying to use such an encrypted directory will fail. + if ! touch $dir/file; then + _notrun "encryption policy '$set_encpolicy_args' is unusable; probably missing kernel crypto API support" + fi + $KEYCTL_PROG clear @s + rm -r $dir +} + _scratch_mkfs_encrypted() { case $FSTYP in -- 2.21.0.593.g511ec345e18-goog ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers ` (2 preceding siblings ...) 2019-04-26 20:41 ` [RFC PATCH 3/7] common/encrypt: support requiring other encryption settings Eric Biggers @ 2019-04-26 20:41 ` Eric Biggers 2019-05-12 12:27 ` Eryu Guan 2019-04-26 20:41 ` [RFC PATCH 5/7] generic: verify ciphertext of v1 encryption policies with AES-256 Eric Biggers ` (3 subsequent siblings) 7 siblings, 1 reply; 14+ messages in thread From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel From: Eric Biggers <ebiggers@google.com> Introduce a function _verify_ciphertext_for_encryption_policy() which verifies the correctness of encryption with the specified settings. Basically, it does the following: 1. If missing any prerequisites, skip the test. 2. Create files in encrypted directories on the scratch device. 3. Unmount the scratch device and compare the actual ciphertext stored on-disk to the ciphertext computed by the fscrypt-crypt-util program. Both file contents and names are verified, and non-default encryption modes are supported. Previously, non-default encryption modes were untested by xfstests. Also, while there's an existing test generic/399 that checks that encrypted contents seem random, it doesn't actually test for correctness, nor does it test filenames encryption. Signed-off-by: Eric Biggers <ebiggers@google.com> --- common/encrypt | 390 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 390 insertions(+) diff --git a/common/encrypt b/common/encrypt index 37f16b94..3e48abc0 100644 --- a/common/encrypt +++ b/common/encrypt @@ -263,3 +263,393 @@ _get_encpolicy() $XFS_IO_PROG -c "get_encpolicy $*" "$file" } + +# Retrieve the encryption nonce of the given inode as a hex string. The nonce +# was randomly generated by the filesystem and isn't exposed directly to +# userspace. But it can be read using the filesystem's debugging tools. +_get_encryption_nonce() +{ + local device=$1 + local inode=$2 + + case $FSTYP in + ext4) + # Use debugfs to dump the special xattr named "c", which is the + # file's fscrypt_context. This produces a line like: + # + # c (28) = 01 01 04 02 00 00 00 00 00 00 00 00 ef bd 18 76 5d f6 41 4e c0 a2 cd 5f 91 29 7e 12 + # + # Then filter it to get just the 16-byte 'nonce' field at the end: + # + # efbd18765df6414ec0a2cd5f91297e12 + # + $DEBUGFS_PROG $device -R "ea_get <$inode> c" 2>>$seqres.full \ + | grep '^c' | sed 's/^.*=//' | tr -d ' \n' | tail -c 32 + ;; + f2fs) + # dump.f2fs prints the fscrypt_context like: + # + # xattr: e_name_index:9 e_name:c e_name_len:1 e_value_size:28 e_value: + # format: 1 + # contents_encryption_mode: 0x1 + # filenames_encryption_mode: 0x4 + # flags: 0x2 + # master_key_descriptor: 0000000000000000 + # nonce: EFBD18765DF6414EC0A2CD5F91297E12 + $DUMP_F2FS_PROG -i $inode $device | awk ' + /\<e_name:c\>/ { found = 1 } + /^nonce:/ && found { + print substr($0, length($0) - 31, 32); + found = 0; + }' + ;; + *) + _notrun "_get_encryption_nonce() isn't implemented on $FSTYP" + ;; + esac +} + +# Require support for _get_encryption_nonce() +_require_get_encryption_nonce_support() +{ + echo "Checking for _get_encryption_nonce() support for $FSTYP" >> $seqres.full + case $FSTYP in + ext4) + _require_command "$DEBUGFS_PROG" debugfs + ;; + f2fs) + _require_command "$DUMP_F2FS_PROG" dump.f2fs + ;; + *) + _notrun "_get_encryption_nonce() isn't implemented on $FSTYP" + ;; + esac +} + +# Retrieve the filename stored on-disk for the given file. +# The name is printed to stdout in binary. +_get_on_disk_filename() +{ + local device=$1 + local inode=$2 + local dir_inode=$3 + + case $FSTYP in + ext4) + # Extract the filename from the debugfs output line like: + # + # 131075 100644 (1) 0 0 0 22-Apr-2019 16:54 \xa2\x85\xb0z\x13\xe9\x09\x86R\xed\xdc\xce\xad\x14d\x19 + # + $DEBUGFS_PROG $device -R "ls -l -r <$dir_inode>" \ + 2>>$seqres.full | perl -ne ' + next if not /^\s*'$inode'\s+/; + s/.*?\d\d:\d\d //; + chomp; + s/\\x([[:xdigit:]]{2})/chr hex $1/eg; + print;' + ;; + f2fs) + # Extract the filename from the dump.f2fs output line like: + # + # i_name [UpkzIPuts9by1oDmE+Ivfw] + # + # The name is base64-encoded, so we have to decode it here. + # + $DUMP_F2FS_PROG $device -i $inode | perl -ne ' + next if not /^i_name\s+\[([A-Za-z0-9+,]+)\]/; + chomp $1; + my @chars = split //, $1; + my $ac = 0; + my $bits = 0; + my $table = join "", (A..Z, a..z, 0..9, "+", ","); + foreach (@chars) { + $ac += index($table, $_) << $bits; + $bits += 6; + if ($bits >= 8) { + print chr($ac & 0xff); + $ac >>= 8; + $bits -= 8; + } + } + if ($ac != 0) { + print STDERR "Invalid base64-encoded string!\n"; + }' + ;; + *) + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" + ;; + esac +} + +# Require support for _get_on_disk_filename() +_require_get_on_disk_filename_support() +{ + echo "Checking for _get_on_disk_filename() support for $FSTYP" >> $seqres.full + case $FSTYP in + ext4) + # Verify that the "ls -l -r" debugfs command is supported and + # hex-encodes non-ASCII characters, rather than using an + # ambiguous escaping method. This requires the e2fsprogs patch + # "debugfs: avoid ambiguity when printing filenames" + # (https://marc.info/?l=linux-ext4&m=155596495624232&w=2). + # TODO: once merged, list the minimum e2fsprogs version here. + _require_command "$DEBUGFS_PROG" debugfs + _scratch_mount + touch $SCRATCH_MNT/$'\xc1' + _scratch_unmount + if ! $DEBUGFS_PROG $SCRATCH_DEV -R "ls -l -r /" 2>&1 \ + | tee -a $seqres.full | grep -E -q '\s+\\xc1\s*$'; then + _notrun "debugfs (e2fsprogs) is too old; doesn't support showing unambiguous on-disk filenames" + fi + ;; + f2fs) + # Verify that dump.f2fs shows encrypted filenames in full. This + # requires the patch "f2fs-tools: improve filename printing" + # (https://sourceforge.net/p/linux-f2fs/mailman/message/36648641/). + # TODO: once merged, list the minimum f2fs-tools version here. + + _require_command "$DUMP_F2FS_PROG" dump.f2fs + _require_command "$KEYCTL_PROG" keyctl + _scratch_mount + _new_session_keyring + + local keydesc=$(_generate_encryption_key) + local dir=$SCRATCH_MNT/test.${FUNCNAME[0]} + local file=$dir/$(perl -e 'print "A" x 255') + mkdir $dir + _set_encpolicy $dir $keydesc + touch $file + local inode=$(stat -c %i $file) + + _scratch_unmount + $KEYCTL_PROG clear @s + + # 255-character filename should result in 340 base64 characters. + if ! $DUMP_F2FS_PROG -i $inode $SCRATCH_DEV \ + | grep -E -q '^i_name[[:space:]]+\[[A-Za-z0-9+,]{340}\]'; then + _notrun "dump.f2fs (f2fs-tools) is too old; doesn't support showing unambiguous on-disk filenames" + fi + ;; + *) + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" + ;; + esac +} + +# Get the file's list of on-disk blocks as a comma-separated list of block +# offsets from the start of the device. "Blocks" are 512 bytes each here. +_get_file_block_list() +{ + local file=$1 + + sync + $XFS_IO_PROG -c fiemap $file | perl -ne ' + next if not /^\s*\d+: \[\d+\.\.\d+\]: (\d+)\.\.(\d+)/; + print $_ . "," foreach $1..$2;' | sed 's/,$//' +} + +# Dump a block list that was previously saved by _get_file_block_list(). +_dump_file_blocks() +{ + local device=$1 + local blocklist=$2 + local block + + for block in $(tr ',' ' ' <<< $blocklist); do + dd if=$device bs=512 count=1 skip=$block status=none + done +} + +_do_verify_ciphertext_for_encryption_policy() +{ + local contents_encryption_mode=$1 + local filenames_encryption_mode=$2 + local policy_flags=$3 + local set_encpolicy_args=$4 + local keydesc=$5 + local raw_key_hex=$6 + local crypt_cmd="src/fscrypt-crypt-util $7" + + local blocksize=$(_get_block_size $SCRATCH_MNT) + local test_contents_files=() + local test_filenames_files=() + local i src dir dst inode blocklist \ + padding_flag padding dir_inode len name f nonce decrypted_name + + # Create files whose encrypted contents we'll verify. For each, save + # the information: (copy of original file, inode number of encrypted + # file, comma-separated block list) into test_contents_files[]. + echo "Creating files for contents verification" >> $seqres.full + i=1 + rm -f $tmp.testfile_* + for src in /dev/zero /dev/urandom; do + head -c $((4 * blocksize)) $src > $tmp.testfile_$i + (( i++ )) + done + dir=$SCRATCH_MNT/encdir + mkdir $dir + _set_encpolicy $dir $keydesc $set_encpolicy_args -f $policy_flags + for src in $tmp.testfile_*; do + dst=$dir/${src##*.} + cp $src $dst + inode=$(stat -c %i $dst) + blocklist=$(_get_file_block_list $dst) + test_contents_files+=("$src $inode $blocklist") + done + + # Create files whose encrypted names we'll verify. For each, save the + # information: (original filename, inode number of encrypted file, inode + # of parent directory, padding amount) into test_filenames_files[]. Try + # each padding amount: 4, 8, 16, or 32 bytes. Also try various filename + # lengths, including boundary cases. Assume NAME_MAX == 255. + echo "Creating files for filenames verification" >> $seqres.full + for padding_flag in 0 1 2 3; do + padding=$((4 << padding_flag)) + dir=$SCRATCH_MNT/encdir.pad$padding + mkdir $dir + dir_inode=$(stat -c %i $dir) + _set_encpolicy $dir $keydesc $set_encpolicy_args \ + -f $((policy_flags | padding_flag)) + for len in 1 3 15 16 17 32 100 254 255; do + name=$(tr -d -C a-zA-Z0-9 < /dev/urandom | head -c $len) + touch $dir/$name + inode=$(stat -c %i $dir/$name) + test_filenames_files+=("$name $inode $dir_inode $padding") + done + done + + # Now unmount the filesystem and verify the ciphertext we just wrote. + _scratch_unmount + + echo "Verifying encrypted file contents" >> $seqres.full + for f in "${test_contents_files[@]}"; do + read -r src inode blocklist <<< "$f" + nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode) + _dump_file_blocks $SCRATCH_DEV $blocklist > $tmp.actual_contents + $crypt_cmd $contents_encryption_mode $raw_key_hex \ + --file-nonce=$nonce --block-size=$blocksize \ + < $src > $tmp.expected_contents + if ! cmp $tmp.expected_contents $tmp.actual_contents; then + _fail "Expected encrypted contents != actual encrypted contents. File: $f" + fi + $crypt_cmd $contents_encryption_mode $raw_key_hex --decrypt \ + --file-nonce=$nonce --block-size=$blocksize \ + < $tmp.actual_contents > $tmp.decrypted_contents + if ! cmp $src $tmp.decrypted_contents; then + _fail "Contents decryption sanity check failed. File: $f" + fi + done + + echo "Verifying encrypted file names" >> $seqres.full + for f in "${test_filenames_files[@]}"; do + read -r name inode dir_inode padding <<< "$f" + nonce=$(_get_encryption_nonce $SCRATCH_DEV $dir_inode) + _get_on_disk_filename $SCRATCH_DEV $inode $dir_inode \ + > $tmp.actual_name + echo -n "$name" | \ + $crypt_cmd $filenames_encryption_mode $raw_key_hex \ + --file-nonce=$nonce --padding=$padding \ + --block-size=255 > $tmp.expected_name + if ! cmp $tmp.expected_name $tmp.actual_name; then + _fail "Expected encrypted filename != actual encrypted filename. File: $f" + fi + $crypt_cmd $filenames_encryption_mode $raw_key_hex --decrypt \ + --file-nonce=$nonce --padding=$padding \ + --block-size=255 < $tmp.actual_name \ + > $tmp.decrypted_name + decrypted_name=$(tr -d '\0' < $tmp.decrypted_name) + if [ "$name" != "$decrypted_name" ]; then + _fail "Filename decryption sanity check failed ($name != $decrypted_name). File: $f" + fi + done +} + +_fscrypt_mode_name_to_num() +{ + local name=$1 + + case "$name" in + AES-256-XTS) echo 1 ;; # FS_ENCRYPTION_MODE_AES_256_XTS + AES-256-CTS-CBC) echo 4 ;; # FS_ENCRYPTION_MODE_AES_256_CTS + AES-128-CBC-ESSIV) echo 5 ;; # FS_ENCRYPTION_MODE_AES_128_CBC + AES-128-CTS-CBC) echo 6 ;; # FS_ENCRYPTION_MODE_AES_128_CTS + Adiantum) echo 9 ;; # FS_ENCRYPTION_MODE_ADIANTUM + *) _fail "Unknown fscrypt mode: $name" ;; + esac +} + +# Verify that file contents and names are encrypted correctly when an encryption +# policy of the specified type is used. +# +# The first two parameters are the contents and filenames encryption modes to +# test. Optionally, also specify 'direct' to test the DIRECT_KEY flag. +_verify_ciphertext_for_encryption_policy() +{ + local contents_encryption_mode=$1 + local filenames_encryption_mode=$2 + local opt + local policy_flags=0 + local set_encpolicy_args="" + local crypt_util_args="" + + shift 2 + for opt; do + case "$opt" in + direct) + if [ $contents_encryption_mode != \ + $filenames_encryption_mode ]; then + _fail "For direct key mode, contents and filenames modes must match" + fi + (( policy_flags |= 0x04 )) # FS_POLICY_FLAG_DIRECT_KEY + ;; + *) + _fail "Unknown option '$opt' passed to ${FUNCNAME[0]}" + ;; + esac + done + local contents_mode_num=$(_fscrypt_mode_name_to_num $contents_encryption_mode) + local filenames_mode_num=$(_fscrypt_mode_name_to_num $filenames_encryption_mode) + + set_encpolicy_args+=" -c $contents_mode_num" + set_encpolicy_args+=" -n $filenames_mode_num" + + if (( policy_flags & 0x04 )); then + crypt_util_args+=" --kdf=none" + else + crypt_util_args+=" --kdf=AES-128-ECB" + fi + set_encpolicy_args=${set_encpolicy_args# } + + _require_scratch_encryption $set_encpolicy_args + _require_test_program "fscrypt-crypt-util" + _require_xfs_io_command "fiemap" + _require_get_encryption_nonce_support + _require_get_on_disk_filename_support + _require_command "$KEYCTL_PROG" keyctl + + echo "Creating encryption-capable filesystem" >> $seqres.full + _scratch_mkfs_encrypted &>> $seqres.full + _scratch_mount + + echo "Generating encryption key" >> $seqres.full + local raw_key=$(_generate_raw_encryption_key) + local keydesc=$(_generate_key_descriptor) + _new_session_keyring + _add_encryption_key $keydesc $raw_key + local raw_key_hex=$(echo "$raw_key" | tr -d '\\x') + + echo + echo -e "Verifying ciphertext with parameters:" + echo -e "\tcontents_encryption_mode: $contents_encryption_mode" + echo -e "\tfilenames_encryption_mode: $filenames_encryption_mode" + [ $# -ne 0 ] && echo -e "\toptions: $*" + + _do_verify_ciphertext_for_encryption_policy \ + "$contents_encryption_mode" \ + "$filenames_encryption_mode" \ + "$policy_flags" \ + "$set_encpolicy_args" \ + "$keydesc" \ + "$raw_key_hex" \ + "$crypt_util_args" +} -- 2.21.0.593.g511ec345e18-goog ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests 2019-04-26 20:41 ` [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests Eric Biggers @ 2019-05-12 12:27 ` Eryu Guan 2019-05-13 19:12 ` Eric Biggers 0 siblings, 1 reply; 14+ messages in thread From: Eryu Guan @ 2019-05-12 12:27 UTC (permalink / raw) To: Eric Biggers; +Cc: linux-fscrypt, linux-ext4, fstests, linux-f2fs-devel On Fri, Apr 26, 2019 at 01:41:50PM -0700, Eric Biggers wrote: > From: Eric Biggers <ebiggers@google.com> > > Introduce a function _verify_ciphertext_for_encryption_policy() which > verifies the correctness of encryption with the specified settings. > > Basically, it does the following: > > 1. If missing any prerequisites, skip the test. > > 2. Create files in encrypted directories on the scratch device. > > 3. Unmount the scratch device and compare the actual ciphertext stored > on-disk to the ciphertext computed by the fscrypt-crypt-util program. > > Both file contents and names are verified, and non-default encryption > modes are supported. Previously, non-default encryption modes were > untested by xfstests. Also, while there's an existing test generic/399 > that checks that encrypted contents seem random, it doesn't actually > test for correctness, nor does it test filenames encryption. > > Signed-off-by: Eric Biggers <ebiggers@google.com> > --- > common/encrypt | 390 +++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 390 insertions(+) > > diff --git a/common/encrypt b/common/encrypt > index 37f16b94..3e48abc0 100644 > --- a/common/encrypt > +++ b/common/encrypt > @@ -263,3 +263,393 @@ _get_encpolicy() > > $XFS_IO_PROG -c "get_encpolicy $*" "$file" > } > + > +# Retrieve the encryption nonce of the given inode as a hex string. The nonce > +# was randomly generated by the filesystem and isn't exposed directly to > +# userspace. But it can be read using the filesystem's debugging tools. > +_get_encryption_nonce() > +{ > + local device=$1 > + local inode=$2 > + > + case $FSTYP in > + ext4) > + # Use debugfs to dump the special xattr named "c", which is the > + # file's fscrypt_context. This produces a line like: > + # > + # c (28) = 01 01 04 02 00 00 00 00 00 00 00 00 ef bd 18 76 5d f6 41 4e c0 a2 cd 5f 91 29 7e 12 > + # > + # Then filter it to get just the 16-byte 'nonce' field at the end: > + # > + # efbd18765df6414ec0a2cd5f91297e12 > + # > + $DEBUGFS_PROG $device -R "ea_get <$inode> c" 2>>$seqres.full \ > + | grep '^c' | sed 's/^.*=//' | tr -d ' \n' | tail -c 32 > + ;; > + f2fs) > + # dump.f2fs prints the fscrypt_context like: > + # > + # xattr: e_name_index:9 e_name:c e_name_len:1 e_value_size:28 e_value: > + # format: 1 > + # contents_encryption_mode: 0x1 > + # filenames_encryption_mode: 0x4 > + # flags: 0x2 > + # master_key_descriptor: 0000000000000000 > + # nonce: EFBD18765DF6414EC0A2CD5F91297E12 > + $DUMP_F2FS_PROG -i $inode $device | awk ' > + /\<e_name:c\>/ { found = 1 } > + /^nonce:/ && found { > + print substr($0, length($0) - 31, 32); > + found = 0; > + }' > + ;; > + *) > + _notrun "_get_encryption_nonce() isn't implemented on $FSTYP" > + ;; No need to _notrun here, _require_get_encryption_nonce_support() should be called first and already does the check in test. Perhaps just echoing an error message to indicate the failure here is fine. > + esac > +} > + > +# Require support for _get_encryption_nonce() > +_require_get_encryption_nonce_support() > +{ > + echo "Checking for _get_encryption_nonce() support for $FSTYP" >> $seqres.full > + case $FSTYP in > + ext4) > + _require_command "$DEBUGFS_PROG" debugfs > + ;; > + f2fs) > + _require_command "$DUMP_F2FS_PROG" dump.f2fs > + ;; > + *) > + _notrun "_get_encryption_nonce() isn't implemented on $FSTYP" > + ;; > + esac > +} > + > +# Retrieve the filename stored on-disk for the given file. > +# The name is printed to stdout in binary. > +_get_on_disk_filename() > +{ > + local device=$1 > + local inode=$2 > + local dir_inode=$3 > + > + case $FSTYP in > + ext4) > + # Extract the filename from the debugfs output line like: > + # > + # 131075 100644 (1) 0 0 0 22-Apr-2019 16:54 \xa2\x85\xb0z\x13\xe9\x09\x86R\xed\xdc\xce\xad\x14d\x19 > + # > + $DEBUGFS_PROG $device -R "ls -l -r <$dir_inode>" \ > + 2>>$seqres.full | perl -ne ' > + next if not /^\s*'$inode'\s+/; > + s/.*?\d\d:\d\d //; > + chomp; > + s/\\x([[:xdigit:]]{2})/chr hex $1/eg; > + print;' > + ;; > + f2fs) > + # Extract the filename from the dump.f2fs output line like: > + # > + # i_name [UpkzIPuts9by1oDmE+Ivfw] > + # > + # The name is base64-encoded, so we have to decode it here. > + # > + $DUMP_F2FS_PROG $device -i $inode | perl -ne ' > + next if not /^i_name\s+\[([A-Za-z0-9+,]+)\]/; > + chomp $1; > + my @chars = split //, $1; > + my $ac = 0; > + my $bits = 0; > + my $table = join "", (A..Z, a..z, 0..9, "+", ","); > + foreach (@chars) { > + $ac += index($table, $_) << $bits; > + $bits += 6; > + if ($bits >= 8) { > + print chr($ac & 0xff); > + $ac >>= 8; > + $bits -= 8; > + } > + } > + if ($ac != 0) { > + print STDERR "Invalid base64-encoded string!\n"; > + }' > + ;; > + *) > + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" Same here, the get_on_disk_filename support is checked by _require_get_on_disk_filename_support(). And looks like this function has nothing to do with fs encryption, move it to common/rc? > + ;; > + esac > +} > + > +# Require support for _get_on_disk_filename() > +_require_get_on_disk_filename_support() > +{ > + echo "Checking for _get_on_disk_filename() support for $FSTYP" >> $seqres.full > + case $FSTYP in > + ext4) > + # Verify that the "ls -l -r" debugfs command is supported and > + # hex-encodes non-ASCII characters, rather than using an > + # ambiguous escaping method. This requires the e2fsprogs patch > + # "debugfs: avoid ambiguity when printing filenames" > + # (https://marc.info/?l=linux-ext4&m=155596495624232&w=2). > + # TODO: once merged, list the minimum e2fsprogs version here. > + _require_command "$DEBUGFS_PROG" debugfs > + _scratch_mount > + touch $SCRATCH_MNT/$'\xc1' > + _scratch_unmount > + if ! $DEBUGFS_PROG $SCRATCH_DEV -R "ls -l -r /" 2>&1 \ > + | tee -a $seqres.full | grep -E -q '\s+\\xc1\s*$'; then > + _notrun "debugfs (e2fsprogs) is too old; doesn't support showing unambiguous on-disk filenames" > + fi > + ;; > + f2fs) > + # Verify that dump.f2fs shows encrypted filenames in full. This > + # requires the patch "f2fs-tools: improve filename printing" > + # (https://sourceforge.net/p/linux-f2fs/mailman/message/36648641/). > + # TODO: once merged, list the minimum f2fs-tools version here. > + > + _require_command "$DUMP_F2FS_PROG" dump.f2fs > + _require_command "$KEYCTL_PROG" keyctl > + _scratch_mount > + _new_session_keyring > + > + local keydesc=$(_generate_encryption_key) > + local dir=$SCRATCH_MNT/test.${FUNCNAME[0]} > + local file=$dir/$(perl -e 'print "A" x 255') > + mkdir $dir > + _set_encpolicy $dir $keydesc > + touch $file > + local inode=$(stat -c %i $file) > + > + _scratch_unmount > + $KEYCTL_PROG clear @s > + > + # 255-character filename should result in 340 base64 characters. > + if ! $DUMP_F2FS_PROG -i $inode $SCRATCH_DEV \ > + | grep -E -q '^i_name[[:space:]]+\[[A-Za-z0-9+,]{340}\]'; then > + _notrun "dump.f2fs (f2fs-tools) is too old; doesn't support showing unambiguous on-disk filenames" > + fi > + ;; > + *) > + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" > + ;; > + esac > +} > + > +# Get the file's list of on-disk blocks as a comma-separated list of block > +# offsets from the start of the device. "Blocks" are 512 bytes each here. > +_get_file_block_list() > +{ > + local file=$1 > + > + sync > + $XFS_IO_PROG -c fiemap $file | perl -ne ' > + next if not /^\s*\d+: \[\d+\.\.\d+\]: (\d+)\.\.(\d+)/; > + print $_ . "," foreach $1..$2;' | sed 's/,$//' > +} > + > +# Dump a block list that was previously saved by _get_file_block_list(). > +_dump_file_blocks() > +{ > + local device=$1 > + local blocklist=$2 > + local block > + > + for block in $(tr ',' ' ' <<< $blocklist); do > + dd if=$device bs=512 count=1 skip=$block status=none > + done > +} Above two functions seem generic enough to be moved to common/rc Thanks, Eryu > + > +_do_verify_ciphertext_for_encryption_policy() > +{ > + local contents_encryption_mode=$1 > + local filenames_encryption_mode=$2 > + local policy_flags=$3 > + local set_encpolicy_args=$4 > + local keydesc=$5 > + local raw_key_hex=$6 > + local crypt_cmd="src/fscrypt-crypt-util $7" > + > + local blocksize=$(_get_block_size $SCRATCH_MNT) > + local test_contents_files=() > + local test_filenames_files=() > + local i src dir dst inode blocklist \ > + padding_flag padding dir_inode len name f nonce decrypted_name > + > + # Create files whose encrypted contents we'll verify. For each, save > + # the information: (copy of original file, inode number of encrypted > + # file, comma-separated block list) into test_contents_files[]. > + echo "Creating files for contents verification" >> $seqres.full > + i=1 > + rm -f $tmp.testfile_* > + for src in /dev/zero /dev/urandom; do > + head -c $((4 * blocksize)) $src > $tmp.testfile_$i > + (( i++ )) > + done > + dir=$SCRATCH_MNT/encdir > + mkdir $dir > + _set_encpolicy $dir $keydesc $set_encpolicy_args -f $policy_flags > + for src in $tmp.testfile_*; do > + dst=$dir/${src##*.} > + cp $src $dst > + inode=$(stat -c %i $dst) > + blocklist=$(_get_file_block_list $dst) > + test_contents_files+=("$src $inode $blocklist") > + done > + > + # Create files whose encrypted names we'll verify. For each, save the > + # information: (original filename, inode number of encrypted file, inode > + # of parent directory, padding amount) into test_filenames_files[]. Try > + # each padding amount: 4, 8, 16, or 32 bytes. Also try various filename > + # lengths, including boundary cases. Assume NAME_MAX == 255. > + echo "Creating files for filenames verification" >> $seqres.full > + for padding_flag in 0 1 2 3; do > + padding=$((4 << padding_flag)) > + dir=$SCRATCH_MNT/encdir.pad$padding > + mkdir $dir > + dir_inode=$(stat -c %i $dir) > + _set_encpolicy $dir $keydesc $set_encpolicy_args \ > + -f $((policy_flags | padding_flag)) > + for len in 1 3 15 16 17 32 100 254 255; do > + name=$(tr -d -C a-zA-Z0-9 < /dev/urandom | head -c $len) > + touch $dir/$name > + inode=$(stat -c %i $dir/$name) > + test_filenames_files+=("$name $inode $dir_inode $padding") > + done > + done > + > + # Now unmount the filesystem and verify the ciphertext we just wrote. > + _scratch_unmount > + > + echo "Verifying encrypted file contents" >> $seqres.full > + for f in "${test_contents_files[@]}"; do > + read -r src inode blocklist <<< "$f" > + nonce=$(_get_encryption_nonce $SCRATCH_DEV $inode) > + _dump_file_blocks $SCRATCH_DEV $blocklist > $tmp.actual_contents > + $crypt_cmd $contents_encryption_mode $raw_key_hex \ > + --file-nonce=$nonce --block-size=$blocksize \ > + < $src > $tmp.expected_contents > + if ! cmp $tmp.expected_contents $tmp.actual_contents; then > + _fail "Expected encrypted contents != actual encrypted contents. File: $f" > + fi > + $crypt_cmd $contents_encryption_mode $raw_key_hex --decrypt \ > + --file-nonce=$nonce --block-size=$blocksize \ > + < $tmp.actual_contents > $tmp.decrypted_contents > + if ! cmp $src $tmp.decrypted_contents; then > + _fail "Contents decryption sanity check failed. File: $f" > + fi > + done > + > + echo "Verifying encrypted file names" >> $seqres.full > + for f in "${test_filenames_files[@]}"; do > + read -r name inode dir_inode padding <<< "$f" > + nonce=$(_get_encryption_nonce $SCRATCH_DEV $dir_inode) > + _get_on_disk_filename $SCRATCH_DEV $inode $dir_inode \ > + > $tmp.actual_name > + echo -n "$name" | \ > + $crypt_cmd $filenames_encryption_mode $raw_key_hex \ > + --file-nonce=$nonce --padding=$padding \ > + --block-size=255 > $tmp.expected_name > + if ! cmp $tmp.expected_name $tmp.actual_name; then > + _fail "Expected encrypted filename != actual encrypted filename. File: $f" > + fi > + $crypt_cmd $filenames_encryption_mode $raw_key_hex --decrypt \ > + --file-nonce=$nonce --padding=$padding \ > + --block-size=255 < $tmp.actual_name \ > + > $tmp.decrypted_name > + decrypted_name=$(tr -d '\0' < $tmp.decrypted_name) > + if [ "$name" != "$decrypted_name" ]; then > + _fail "Filename decryption sanity check failed ($name != $decrypted_name). File: $f" > + fi > + done > +} > + > +_fscrypt_mode_name_to_num() > +{ > + local name=$1 > + > + case "$name" in > + AES-256-XTS) echo 1 ;; # FS_ENCRYPTION_MODE_AES_256_XTS > + AES-256-CTS-CBC) echo 4 ;; # FS_ENCRYPTION_MODE_AES_256_CTS > + AES-128-CBC-ESSIV) echo 5 ;; # FS_ENCRYPTION_MODE_AES_128_CBC > + AES-128-CTS-CBC) echo 6 ;; # FS_ENCRYPTION_MODE_AES_128_CTS > + Adiantum) echo 9 ;; # FS_ENCRYPTION_MODE_ADIANTUM > + *) _fail "Unknown fscrypt mode: $name" ;; > + esac > +} > + > +# Verify that file contents and names are encrypted correctly when an encryption > +# policy of the specified type is used. > +# > +# The first two parameters are the contents and filenames encryption modes to > +# test. Optionally, also specify 'direct' to test the DIRECT_KEY flag. > +_verify_ciphertext_for_encryption_policy() > +{ > + local contents_encryption_mode=$1 > + local filenames_encryption_mode=$2 > + local opt > + local policy_flags=0 > + local set_encpolicy_args="" > + local crypt_util_args="" > + > + shift 2 > + for opt; do > + case "$opt" in > + direct) > + if [ $contents_encryption_mode != \ > + $filenames_encryption_mode ]; then > + _fail "For direct key mode, contents and filenames modes must match" > + fi > + (( policy_flags |= 0x04 )) # FS_POLICY_FLAG_DIRECT_KEY > + ;; > + *) > + _fail "Unknown option '$opt' passed to ${FUNCNAME[0]}" > + ;; > + esac > + done > + local contents_mode_num=$(_fscrypt_mode_name_to_num $contents_encryption_mode) > + local filenames_mode_num=$(_fscrypt_mode_name_to_num $filenames_encryption_mode) > + > + set_encpolicy_args+=" -c $contents_mode_num" > + set_encpolicy_args+=" -n $filenames_mode_num" > + > + if (( policy_flags & 0x04 )); then > + crypt_util_args+=" --kdf=none" > + else > + crypt_util_args+=" --kdf=AES-128-ECB" > + fi > + set_encpolicy_args=${set_encpolicy_args# } > + > + _require_scratch_encryption $set_encpolicy_args > + _require_test_program "fscrypt-crypt-util" > + _require_xfs_io_command "fiemap" > + _require_get_encryption_nonce_support > + _require_get_on_disk_filename_support > + _require_command "$KEYCTL_PROG" keyctl > + > + echo "Creating encryption-capable filesystem" >> $seqres.full > + _scratch_mkfs_encrypted &>> $seqres.full > + _scratch_mount > + > + echo "Generating encryption key" >> $seqres.full > + local raw_key=$(_generate_raw_encryption_key) > + local keydesc=$(_generate_key_descriptor) > + _new_session_keyring > + _add_encryption_key $keydesc $raw_key > + local raw_key_hex=$(echo "$raw_key" | tr -d '\\x') > + > + echo > + echo -e "Verifying ciphertext with parameters:" > + echo -e "\tcontents_encryption_mode: $contents_encryption_mode" > + echo -e "\tfilenames_encryption_mode: $filenames_encryption_mode" > + [ $# -ne 0 ] && echo -e "\toptions: $*" > + > + _do_verify_ciphertext_for_encryption_policy \ > + "$contents_encryption_mode" \ > + "$filenames_encryption_mode" \ > + "$policy_flags" \ > + "$set_encpolicy_args" \ > + "$keydesc" \ > + "$raw_key_hex" \ > + "$crypt_util_args" > +} > -- > 2.21.0.593.g511ec345e18-goog > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests 2019-05-12 12:27 ` Eryu Guan @ 2019-05-13 19:12 ` Eric Biggers 2019-05-14 2:20 ` Eryu Guan 0 siblings, 1 reply; 14+ messages in thread From: Eric Biggers @ 2019-05-13 19:12 UTC (permalink / raw) To: Eryu Guan; +Cc: linux-fscrypt, linux-ext4, fstests, linux-f2fs-devel Hi Eryu, On Sun, May 12, 2019 at 08:27:03PM +0800, Eryu Guan wrote: > > +# Retrieve the filename stored on-disk for the given file. > > +# The name is printed to stdout in binary. > > +_get_on_disk_filename() > > +{ > > + local device=$1 > > + local inode=$2 > > + local dir_inode=$3 > > + > > + case $FSTYP in > > + ext4) > > + # Extract the filename from the debugfs output line like: > > + # > > + # 131075 100644 (1) 0 0 0 22-Apr-2019 16:54 \xa2\x85\xb0z\x13\xe9\x09\x86R\xed\xdc\xce\xad\x14d\x19 > > + # > > + $DEBUGFS_PROG $device -R "ls -l -r <$dir_inode>" \ > > + 2>>$seqres.full | perl -ne ' > > + next if not /^\s*'$inode'\s+/; > > + s/.*?\d\d:\d\d //; > > + chomp; > > + s/\\x([[:xdigit:]]{2})/chr hex $1/eg; > > + print;' > > + ;; > > + f2fs) > > + # Extract the filename from the dump.f2fs output line like: > > + # > > + # i_name [UpkzIPuts9by1oDmE+Ivfw] > > + # > > + # The name is base64-encoded, so we have to decode it here. > > + # > > + $DUMP_F2FS_PROG $device -i $inode | perl -ne ' > > + next if not /^i_name\s+\[([A-Za-z0-9+,]+)\]/; > > + chomp $1; > > + my @chars = split //, $1; > > + my $ac = 0; > > + my $bits = 0; > > + my $table = join "", (A..Z, a..z, 0..9, "+", ","); > > + foreach (@chars) { > > + $ac += index($table, $_) << $bits; > > + $bits += 6; > > + if ($bits >= 8) { > > + print chr($ac & 0xff); > > + $ac >>= 8; > > + $bits -= 8; > > + } > > + } > > + if ($ac != 0) { > > + print STDERR "Invalid base64-encoded string!\n"; > > + }' > > + ;; > > + *) > > + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" > > And looks like this function has nothing to do with fs encryption, move it to > common/rc? For ext4 that's true, but for f2fs the name is assumed to be base64-encoded, which f2fs-tools only does for encrypted filenames. I'll update the comment to clarify that the function assumes the directory is encrypted. > > > + ;; > > + esac > > +} > > + > > +# Require support for _get_on_disk_filename() > > +_require_get_on_disk_filename_support() > > +{ > > + echo "Checking for _get_on_disk_filename() support for $FSTYP" >> $seqres.full > > + case $FSTYP in > > + ext4) > > + # Verify that the "ls -l -r" debugfs command is supported and > > + # hex-encodes non-ASCII characters, rather than using an > > + # ambiguous escaping method. This requires the e2fsprogs patch > > + # "debugfs: avoid ambiguity when printing filenames" > > + # (https://marc.info/?l=linux-ext4&m=155596495624232&w=2). > > + # TODO: once merged, list the minimum e2fsprogs version here. > > + _require_command "$DEBUGFS_PROG" debugfs > > + _scratch_mount > > + touch $SCRATCH_MNT/$'\xc1' > > + _scratch_unmount > > + if ! $DEBUGFS_PROG $SCRATCH_DEV -R "ls -l -r /" 2>&1 \ > > + | tee -a $seqres.full | grep -E -q '\s+\\xc1\s*$'; then > > + _notrun "debugfs (e2fsprogs) is too old; doesn't support showing unambiguous on-disk filenames" > > + fi > > + ;; > > + f2fs) > > + # Verify that dump.f2fs shows encrypted filenames in full. This > > + # requires the patch "f2fs-tools: improve filename printing" > > + # (https://sourceforge.net/p/linux-f2fs/mailman/message/36648641/). > > + # TODO: once merged, list the minimum f2fs-tools version here. > > + > > + _require_command "$DUMP_F2FS_PROG" dump.f2fs > > + _require_command "$KEYCTL_PROG" keyctl > > + _scratch_mount > > + _new_session_keyring > > + > > + local keydesc=$(_generate_encryption_key) > > + local dir=$SCRATCH_MNT/test.${FUNCNAME[0]} > > + local file=$dir/$(perl -e 'print "A" x 255') > > + mkdir $dir > > + _set_encpolicy $dir $keydesc > > + touch $file > > + local inode=$(stat -c %i $file) > > + > > + _scratch_unmount > > + $KEYCTL_PROG clear @s > > + > > + # 255-character filename should result in 340 base64 characters. > > + if ! $DUMP_F2FS_PROG -i $inode $SCRATCH_DEV \ > > + | grep -E -q '^i_name[[:space:]]+\[[A-Za-z0-9+,]{340}\]'; then > > + _notrun "dump.f2fs (f2fs-tools) is too old; doesn't support showing unambiguous on-disk filenames" > > + fi > > + ;; > > + *) > > + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" > > + ;; > > + esac > > +} > > + > > +# Get the file's list of on-disk blocks as a comma-separated list of block > > +# offsets from the start of the device. "Blocks" are 512 bytes each here. > > +_get_file_block_list() > > +{ > > + local file=$1 > > + > > + sync > > + $XFS_IO_PROG -c fiemap $file | perl -ne ' > > + next if not /^\s*\d+: \[\d+\.\.\d+\]: (\d+)\.\.(\d+)/; > > + print $_ . "," foreach $1..$2;' | sed 's/,$//' > > +} > > + > > +# Dump a block list that was previously saved by _get_file_block_list(). > > +_dump_file_blocks() > > +{ > > + local device=$1 > > + local blocklist=$2 > > + local block > > + > > + for block in $(tr ',' ' ' <<< $blocklist); do > > + dd if=$device bs=512 count=1 skip=$block status=none > > + done > > +} > > Above two functions seem generic enough to be moved to common/rc I feel that would be premature because common/rc is kind of bloated, and there's a good chance these functions will only ever be used for encryption tests. Normally, xfstests only test for user-visible behavior, so tests just 'cat' the file contents, or 'ls' the filenames. The encryption tests are somewhat special in that they really care about what's *actually* stored on-disk. So I think that common/encrypt is the most logical location for now. But I don't feel too strongly, and I can move it if you prefer. Thanks for the review! - Eric ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests 2019-05-13 19:12 ` Eric Biggers @ 2019-05-14 2:20 ` Eryu Guan 0 siblings, 0 replies; 14+ messages in thread From: Eryu Guan @ 2019-05-14 2:20 UTC (permalink / raw) To: Eric Biggers; +Cc: linux-fscrypt, linux-ext4, fstests, linux-f2fs-devel On Mon, May 13, 2019 at 12:12:05PM -0700, Eric Biggers wrote: > Hi Eryu, > > On Sun, May 12, 2019 at 08:27:03PM +0800, Eryu Guan wrote: > > > +# Retrieve the filename stored on-disk for the given file. > > > +# The name is printed to stdout in binary. > > > +_get_on_disk_filename() > > > +{ > > > + local device=$1 > > > + local inode=$2 > > > + local dir_inode=$3 > > > + > > > + case $FSTYP in > > > + ext4) > > > + # Extract the filename from the debugfs output line like: > > > + # > > > + # 131075 100644 (1) 0 0 0 22-Apr-2019 16:54 \xa2\x85\xb0z\x13\xe9\x09\x86R\xed\xdc\xce\xad\x14d\x19 > > > + # > > > + $DEBUGFS_PROG $device -R "ls -l -r <$dir_inode>" \ > > > + 2>>$seqres.full | perl -ne ' > > > + next if not /^\s*'$inode'\s+/; > > > + s/.*?\d\d:\d\d //; > > > + chomp; > > > + s/\\x([[:xdigit:]]{2})/chr hex $1/eg; > > > + print;' > > > + ;; > > > + f2fs) > > > + # Extract the filename from the dump.f2fs output line like: > > > + # > > > + # i_name [UpkzIPuts9by1oDmE+Ivfw] > > > + # > > > + # The name is base64-encoded, so we have to decode it here. > > > + # > > > + $DUMP_F2FS_PROG $device -i $inode | perl -ne ' > > > + next if not /^i_name\s+\[([A-Za-z0-9+,]+)\]/; > > > + chomp $1; > > > + my @chars = split //, $1; > > > + my $ac = 0; > > > + my $bits = 0; > > > + my $table = join "", (A..Z, a..z, 0..9, "+", ","); > > > + foreach (@chars) { > > > + $ac += index($table, $_) << $bits; > > > + $bits += 6; > > > + if ($bits >= 8) { > > > + print chr($ac & 0xff); > > > + $ac >>= 8; > > > + $bits -= 8; > > > + } > > > + } > > > + if ($ac != 0) { > > > + print STDERR "Invalid base64-encoded string!\n"; > > > + }' > > > + ;; > > > + *) > > > + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" > > > > And looks like this function has nothing to do with fs encryption, move it to > > common/rc? > > For ext4 that's true, but for f2fs the name is assumed to be base64-encoded, > which f2fs-tools only does for encrypted filenames. I'll update the comment to > clarify that the function assumes the directory is encrypted. > > > > > > + ;; > > > + esac > > > +} > > > + > > > +# Require support for _get_on_disk_filename() > > > +_require_get_on_disk_filename_support() > > > +{ > > > + echo "Checking for _get_on_disk_filename() support for $FSTYP" >> $seqres.full > > > + case $FSTYP in > > > + ext4) > > > + # Verify that the "ls -l -r" debugfs command is supported and > > > + # hex-encodes non-ASCII characters, rather than using an > > > + # ambiguous escaping method. This requires the e2fsprogs patch > > > + # "debugfs: avoid ambiguity when printing filenames" > > > + # (https://marc.info/?l=linux-ext4&m=155596495624232&w=2). > > > + # TODO: once merged, list the minimum e2fsprogs version here. > > > + _require_command "$DEBUGFS_PROG" debugfs > > > + _scratch_mount > > > + touch $SCRATCH_MNT/$'\xc1' > > > + _scratch_unmount > > > + if ! $DEBUGFS_PROG $SCRATCH_DEV -R "ls -l -r /" 2>&1 \ > > > + | tee -a $seqres.full | grep -E -q '\s+\\xc1\s*$'; then > > > + _notrun "debugfs (e2fsprogs) is too old; doesn't support showing unambiguous on-disk filenames" > > > + fi > > > + ;; > > > + f2fs) > > > + # Verify that dump.f2fs shows encrypted filenames in full. This > > > + # requires the patch "f2fs-tools: improve filename printing" > > > + # (https://sourceforge.net/p/linux-f2fs/mailman/message/36648641/). > > > + # TODO: once merged, list the minimum f2fs-tools version here. > > > + > > > + _require_command "$DUMP_F2FS_PROG" dump.f2fs > > > + _require_command "$KEYCTL_PROG" keyctl > > > + _scratch_mount > > > + _new_session_keyring > > > + > > > + local keydesc=$(_generate_encryption_key) > > > + local dir=$SCRATCH_MNT/test.${FUNCNAME[0]} > > > + local file=$dir/$(perl -e 'print "A" x 255') > > > + mkdir $dir > > > + _set_encpolicy $dir $keydesc > > > + touch $file > > > + local inode=$(stat -c %i $file) > > > + > > > + _scratch_unmount > > > + $KEYCTL_PROG clear @s > > > + > > > + # 255-character filename should result in 340 base64 characters. > > > + if ! $DUMP_F2FS_PROG -i $inode $SCRATCH_DEV \ > > > + | grep -E -q '^i_name[[:space:]]+\[[A-Za-z0-9+,]{340}\]'; then > > > + _notrun "dump.f2fs (f2fs-tools) is too old; doesn't support showing unambiguous on-disk filenames" > > > + fi > > > + ;; > > > + *) > > > + _notrun "_get_on_disk_filename() isn't implemented on $FSTYP" > > > + ;; > > > + esac > > > +} > > > + > > > +# Get the file's list of on-disk blocks as a comma-separated list of block > > > +# offsets from the start of the device. "Blocks" are 512 bytes each here. > > > +_get_file_block_list() > > > +{ > > > + local file=$1 > > > + > > > + sync > > > + $XFS_IO_PROG -c fiemap $file | perl -ne ' > > > + next if not /^\s*\d+: \[\d+\.\.\d+\]: (\d+)\.\.(\d+)/; > > > + print $_ . "," foreach $1..$2;' | sed 's/,$//' > > > +} > > > + > > > +# Dump a block list that was previously saved by _get_file_block_list(). > > > +_dump_file_blocks() > > > +{ > > > + local device=$1 > > > + local blocklist=$2 > > > + local block > > > + > > > + for block in $(tr ',' ' ' <<< $blocklist); do > > > + dd if=$device bs=512 count=1 skip=$block status=none > > > + done > > > +} > > > > Above two functions seem generic enough to be moved to common/rc > > I feel that would be premature because common/rc is kind of bloated, and there's > a good chance these functions will only ever be used for encryption tests. > Normally, xfstests only test for user-visible behavior, so tests just 'cat' the > file contents, or 'ls' the filenames. The encryption tests are somewhat special > in that they really care about what's *actually* stored on-disk. > > So I think that common/encrypt is the most logical location for now. But I Yeah, that makes sense to me. Perhaps the functions should be renamed to reflect that they're encryption-related? Thanks, Eryu > don't feel too strongly, and I can move it if you prefer. > > Thanks for the review! > > - Eric ^ permalink raw reply [flat|nested] 14+ messages in thread
* [RFC PATCH 5/7] generic: verify ciphertext of v1 encryption policies with AES-256 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers ` (3 preceding siblings ...) 2019-04-26 20:41 ` [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests Eric Biggers @ 2019-04-26 20:41 ` Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 6/7] generic: verify ciphertext of v1 encryption policies with AES-128 Eric Biggers ` (2 subsequent siblings) 7 siblings, 0 replies; 14+ messages in thread From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel From: Eric Biggers <ebiggers@google.com> Verify ciphertext for v1 encryption policies that use AES-256-XTS to encrypt file contents and AES-256-CTS-CBC to encrypt file names. Signed-off-by: Eric Biggers <ebiggers@google.com> --- tests/generic/700 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/generic/700.out | 5 +++++ tests/generic/group | 1 + 3 files changed, 47 insertions(+) create mode 100755 tests/generic/700 create mode 100644 tests/generic/700.out diff --git a/tests/generic/700 b/tests/generic/700 new file mode 100755 index 00000000..c6dc8b90 --- /dev/null +++ b/tests/generic/700 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2019 Google LLC +# +# FS QA Test generic/700 +# +# Verify ciphertext for v1 encryption policies that use AES-256-XTS to encrypt +# file contents and AES-256-CTS-CBC to encrypt file names. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/encrypt + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux + +_verify_ciphertext_for_encryption_policy AES-256-XTS AES-256-CTS-CBC + +# success, all done +status=0 +exit diff --git a/tests/generic/700.out b/tests/generic/700.out new file mode 100644 index 00000000..ea2de143 --- /dev/null +++ b/tests/generic/700.out @@ -0,0 +1,5 @@ +QA output created by 700 + +Verifying ciphertext with parameters: + contents_encryption_mode: AES-256-XTS + filenames_encryption_mode: AES-256-CTS-CBC diff --git a/tests/generic/group b/tests/generic/group index 40deb4d0..22e5c028 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -547,3 +547,4 @@ 542 auto quick clone 543 auto quick clone 544 auto quick clone +700 auto quick encrypt -- 2.21.0.593.g511ec345e18-goog ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [RFC PATCH 6/7] generic: verify ciphertext of v1 encryption policies with AES-128 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers ` (4 preceding siblings ...) 2019-04-26 20:41 ` [RFC PATCH 5/7] generic: verify ciphertext of v1 encryption policies with AES-256 Eric Biggers @ 2019-04-26 20:41 ` Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 7/7] generic: verify ciphertext of v1 encryption policies with Adiantum Eric Biggers 2019-05-06 15:57 ` [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers 7 siblings, 0 replies; 14+ messages in thread From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel From: Eric Biggers <ebiggers@google.com> Verify ciphertext for v1 encryption policies that use AES-128-CBC-ESSIV to encrypt file contents and AES-128-CTS-CBC to encrypt file names. Signed-off-by: Eric Biggers <ebiggers@google.com> --- tests/generic/701 | 41 +++++++++++++++++++++++++++++++++++++++++ tests/generic/701.out | 5 +++++ tests/generic/group | 1 + 3 files changed, 47 insertions(+) create mode 100755 tests/generic/701 create mode 100644 tests/generic/701.out diff --git a/tests/generic/701 b/tests/generic/701 new file mode 100755 index 00000000..d477f5bd --- /dev/null +++ b/tests/generic/701 @@ -0,0 +1,41 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2019 Google LLC +# +# FS QA Test generic/701 +# +# Verify ciphertext for v1 encryption policies that use AES-128-CBC-ESSIV to +# encrypt file contents and AES-128-CTS-CBC to encrypt file names. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/encrypt + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux + +_verify_ciphertext_for_encryption_policy AES-128-CBC-ESSIV AES-128-CTS-CBC + +# success, all done +status=0 +exit diff --git a/tests/generic/701.out b/tests/generic/701.out new file mode 100644 index 00000000..cfb6c924 --- /dev/null +++ b/tests/generic/701.out @@ -0,0 +1,5 @@ +QA output created by 701 + +Verifying ciphertext with parameters: + contents_encryption_mode: AES-128-CBC-ESSIV + filenames_encryption_mode: AES-128-CTS-CBC diff --git a/tests/generic/group b/tests/generic/group index 22e5c028..6bb793cb 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -548,3 +548,4 @@ 543 auto quick clone 544 auto quick clone 700 auto quick encrypt +701 auto quick encrypt -- 2.21.0.593.g511ec345e18-goog ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [RFC PATCH 7/7] generic: verify ciphertext of v1 encryption policies with Adiantum 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers ` (5 preceding siblings ...) 2019-04-26 20:41 ` [RFC PATCH 6/7] generic: verify ciphertext of v1 encryption policies with AES-128 Eric Biggers @ 2019-04-26 20:41 ` Eric Biggers 2019-05-06 15:57 ` [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers 7 siblings, 0 replies; 14+ messages in thread From: Eric Biggers @ 2019-04-26 20:41 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel From: Eric Biggers <ebiggers@google.com> Verify ciphertext for v1 encryption policies that use Adiantum to encrypt file contents and file names. Signed-off-by: Eric Biggers <ebiggers@google.com> --- tests/generic/702 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/702.out | 10 ++++++++++ tests/generic/group | 1 + 3 files changed, 54 insertions(+) create mode 100755 tests/generic/702 create mode 100644 tests/generic/702.out diff --git a/tests/generic/702 b/tests/generic/702 new file mode 100755 index 00000000..c7ac7be4 --- /dev/null +++ b/tests/generic/702 @@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2019 Google LLC +# +# FS QA Test generic/702 +# +# Verify ciphertext for v1 encryption policies that use Adiantum to encrypt file +# contents and file names. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/encrypt + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs generic +_supported_os Linux + +# Test both with and without the DIRECT_KEY flag. +_verify_ciphertext_for_encryption_policy Adiantum Adiantum +_verify_ciphertext_for_encryption_policy Adiantum Adiantum direct + +# success, all done +status=0 +exit diff --git a/tests/generic/702.out b/tests/generic/702.out new file mode 100644 index 00000000..0dd80dae --- /dev/null +++ b/tests/generic/702.out @@ -0,0 +1,10 @@ +QA output created by 702 + +Verifying ciphertext with parameters: + contents_encryption_mode: Adiantum + filenames_encryption_mode: Adiantum + +Verifying ciphertext with parameters: + contents_encryption_mode: Adiantum + filenames_encryption_mode: Adiantum + options: direct diff --git a/tests/generic/group b/tests/generic/group index 6bb793cb..f6672b93 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -549,3 +549,4 @@ 544 auto quick clone 700 auto quick encrypt 701 auto quick encrypt +702 auto quick encrypt -- 2.21.0.593.g511ec345e18-goog ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers ` (6 preceding siblings ...) 2019-04-26 20:41 ` [RFC PATCH 7/7] generic: verify ciphertext of v1 encryption policies with Adiantum Eric Biggers @ 2019-05-06 15:57 ` Eric Biggers 2019-05-12 12:58 ` Eryu Guan 7 siblings, 1 reply; 14+ messages in thread From: Eric Biggers @ 2019-05-06 15:57 UTC (permalink / raw) To: fstests; +Cc: linux-fscrypt, linux-ext4, linux-f2fs-devel On Fri, Apr 26, 2019 at 01:41:46PM -0700, Eric Biggers wrote: > Hello, > > This series adds xfstests which verify that encrypted contents and > filenames on ext4 and f2fs are actually correct, i.e. that the > encryption uses the correct algorithms, keys, IVs, and padding amounts. > The new tests work by creating encrypted files, unmounting the > filesystem, reading the ciphertext from disk using dd and debugfs or > dump.f2fs, and then comparing it against ciphertext computed > independently by a new test program that implements the same algorithms. > > These tests are important because: > > - The whole point of file encryption is that the files are actually > encrypted correctly on-disk. Except for generic/399, current xfstests > only tests the filesystem semantics, not the actual encryption. > generic/399 only tests for incompressibility of encrypted file > contents using one particular encryption setting, which isn't much. > > - fscrypt now supports 4 main combinations of encryption settings, > rather than 1 as it did originally. This may be doubled to 8 soon > (https://patchwork.kernel.org/patch/10908153/). We should test all > settings. And without tests, even if the initial implementation is > correct, breakage in one specific setting could go undetected. > > - Though Linux's crypto API has self-tests, these only test the > algorithms themselves, not how they are used, e.g. by fscrypt. > > Patch 1 is a cleanup patch. Patches 2-4 add the common helpers for > ciphertext verification tests. Patches 5-7 add the actual tests. > > These tests require e2fsprogs and f2fs-tools patches I recently sent out > to fix printing encrypted filenames. So, this series might not be > suitable for merging into mainline xfstests until those patches are > applied. Regardless, comments are appreciated. The needed patches are: > > debugfs: avoid ambiguity when printing filenames (https://marc.info/?l=linux-ext4&m=155596495624232&w=2) > f2fs-tools: improve filename printing (https://sourceforge.net/p/linux-f2fs/mailman/message/36648641/) > > This series can also be retrieved from git at > https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/xfstests-dev.git > branch "ciphertext-verification". > > I also have patches on top of this series which verify the ciphertext > produced from v2 encryption policies, which are proposed by my kernel > patch series "fscrypt: key management improvements" > (https://patchwork.kernel.org/cover/10908107/). v2 encryption policies > will use a different key derivation function, and thus their ciphertext > will be different. These additional patches can be found at branch > "fscrypt-key-mgmt-improvements" of my git repo above. But I've arranged > things such that this shorter series can potentially be applied earlier, > to test what's in the kernel now. > > Eric Biggers (7): > common/encrypt: introduce helpers for set_encpolicy and get_encpolicy > fscrypt-crypt-util: add utility for reproducing fscrypt encrypted data > common/encrypt: support requiring other encryption settings > common/encrypt: add helper for ciphertext verification tests > generic: verify ciphertext of v1 encryption policies with AES-256 > generic: verify ciphertext of v1 encryption policies with AES-128 > generic: verify ciphertext of v1 encryption policies with Adiantum > > .gitignore | 1 + > common/encrypt | 482 ++++++++++- > src/Makefile | 3 +- > src/fscrypt-crypt-util.c | 1645 ++++++++++++++++++++++++++++++++++++++ > tests/ext4/024 | 3 +- > tests/generic/395 | 28 +- > tests/generic/395.out | 2 +- > tests/generic/396 | 15 +- > tests/generic/397 | 3 +- > tests/generic/398 | 5 +- > tests/generic/399 | 3 +- > tests/generic/419 | 3 +- > tests/generic/421 | 3 +- > tests/generic/429 | 3 +- > tests/generic/435 | 3 +- > tests/generic/440 | 5 +- > tests/generic/700 | 41 + > tests/generic/700.out | 5 + > tests/generic/701 | 41 + > tests/generic/701.out | 5 + > tests/generic/702 | 43 + > tests/generic/702.out | 10 + > tests/generic/group | 3 + > 23 files changed, 2308 insertions(+), 47 deletions(-) > create mode 100644 src/fscrypt-crypt-util.c > create mode 100755 tests/generic/700 > create mode 100644 tests/generic/700.out > create mode 100755 tests/generic/701 > create mode 100644 tests/generic/701.out > create mode 100755 tests/generic/702 > create mode 100644 tests/generic/702.out > > -- > 2.21.0.593.g511ec345e18-goog > Any comments on this? FYI, the e2fsprogs patch that these tests need was applied. I'm still waiting for the f2fs-tools patch. - Eric ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames 2019-05-06 15:57 ` [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers @ 2019-05-12 12:58 ` Eryu Guan 0 siblings, 0 replies; 14+ messages in thread From: Eryu Guan @ 2019-05-12 12:58 UTC (permalink / raw) To: Eric Biggers; +Cc: linux-fscrypt, linux-ext4, fstests, linux-f2fs-devel On Mon, May 06, 2019 at 08:57:22AM -0700, Eric Biggers wrote: > On Fri, Apr 26, 2019 at 01:41:46PM -0700, Eric Biggers wrote: > > Hello, > > > > This series adds xfstests which verify that encrypted contents and > > filenames on ext4 and f2fs are actually correct, i.e. that the > > encryption uses the correct algorithms, keys, IVs, and padding amounts. > > The new tests work by creating encrypted files, unmounting the > > filesystem, reading the ciphertext from disk using dd and debugfs or > > dump.f2fs, and then comparing it against ciphertext computed > > independently by a new test program that implements the same algorithms. > > > > These tests are important because: > > > > - The whole point of file encryption is that the files are actually > > encrypted correctly on-disk. Except for generic/399, current xfstests > > only tests the filesystem semantics, not the actual encryption. > > generic/399 only tests for incompressibility of encrypted file > > contents using one particular encryption setting, which isn't much. > > > > - fscrypt now supports 4 main combinations of encryption settings, > > rather than 1 as it did originally. This may be doubled to 8 soon > > (https://patchwork.kernel.org/patch/10908153/). We should test all > > settings. And without tests, even if the initial implementation is > > correct, breakage in one specific setting could go undetected. > > > > - Though Linux's crypto API has self-tests, these only test the > > algorithms themselves, not how they are used, e.g. by fscrypt. > > > > Patch 1 is a cleanup patch. Patches 2-4 add the common helpers for > > ciphertext verification tests. Patches 5-7 add the actual tests. > > > > These tests require e2fsprogs and f2fs-tools patches I recently sent out > > to fix printing encrypted filenames. So, this series might not be > > suitable for merging into mainline xfstests until those patches are > > applied. Regardless, comments are appreciated. The needed patches are: > > > > debugfs: avoid ambiguity when printing filenames (https://marc.info/?l=linux-ext4&m=155596495624232&w=2) > > f2fs-tools: improve filename printing (https://sourceforge.net/p/linux-f2fs/mailman/message/36648641/) > > > > This series can also be retrieved from git at > > https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/xfstests-dev.git > > branch "ciphertext-verification". > > > > I also have patches on top of this series which verify the ciphertext > > produced from v2 encryption policies, which are proposed by my kernel > > patch series "fscrypt: key management improvements" > > (https://patchwork.kernel.org/cover/10908107/). v2 encryption policies > > will use a different key derivation function, and thus their ciphertext > > will be different. These additional patches can be found at branch > > "fscrypt-key-mgmt-improvements" of my git repo above. But I've arranged > > things such that this shorter series can potentially be applied earlier, > > to test what's in the kernel now. > > > > Eric Biggers (7): > > common/encrypt: introduce helpers for set_encpolicy and get_encpolicy > > fscrypt-crypt-util: add utility for reproducing fscrypt encrypted data > > common/encrypt: support requiring other encryption settings > > common/encrypt: add helper for ciphertext verification tests > > generic: verify ciphertext of v1 encryption policies with AES-256 > > generic: verify ciphertext of v1 encryption policies with AES-128 > > generic: verify ciphertext of v1 encryption policies with Adiantum > > > > .gitignore | 1 + > > common/encrypt | 482 ++++++++++- > > src/Makefile | 3 +- > > src/fscrypt-crypt-util.c | 1645 ++++++++++++++++++++++++++++++++++++++ > > tests/ext4/024 | 3 +- > > tests/generic/395 | 28 +- > > tests/generic/395.out | 2 +- > > tests/generic/396 | 15 +- > > tests/generic/397 | 3 +- > > tests/generic/398 | 5 +- > > tests/generic/399 | 3 +- > > tests/generic/419 | 3 +- > > tests/generic/421 | 3 +- > > tests/generic/429 | 3 +- > > tests/generic/435 | 3 +- > > tests/generic/440 | 5 +- > > tests/generic/700 | 41 + > > tests/generic/700.out | 5 + > > tests/generic/701 | 41 + > > tests/generic/701.out | 5 + > > tests/generic/702 | 43 + > > tests/generic/702.out | 10 + > > tests/generic/group | 3 + > > 23 files changed, 2308 insertions(+), 47 deletions(-) > > create mode 100644 src/fscrypt-crypt-util.c > > create mode 100755 tests/generic/700 > > create mode 100644 tests/generic/700.out > > create mode 100755 tests/generic/701 > > create mode 100644 tests/generic/701.out > > create mode 100755 tests/generic/702 > > create mode 100644 tests/generic/702.out > > > > -- > > 2.21.0.593.g511ec345e18-goog > > > > Any comments on this? Sorry for the late review, I went through the patches and they look fine to me over all from fstests perspective, I replied a few minor issues to individual patches. It'd be great if ext4 and/or f2fs folks could help review the tests as well. Thanks, Eryu > > FYI, the e2fsprogs patch that these tests need was applied. > > I'm still waiting for the f2fs-tools patch. > > - Eric ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2019-05-14 2:20 UTC | newest] Thread overview: 14+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-04-26 20:41 [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 1/7] common/encrypt: introduce helpers for set_encpolicy and get_encpolicy Eric Biggers 2019-05-12 12:21 ` Eryu Guan 2019-04-26 20:41 ` [RFC PATCH 2/7] fscrypt-crypt-util: add utility for reproducing fscrypt encrypted data Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 3/7] common/encrypt: support requiring other encryption settings Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 4/7] common/encrypt: add helper for ciphertext verification tests Eric Biggers 2019-05-12 12:27 ` Eryu Guan 2019-05-13 19:12 ` Eric Biggers 2019-05-14 2:20 ` Eryu Guan 2019-04-26 20:41 ` [RFC PATCH 5/7] generic: verify ciphertext of v1 encryption policies with AES-256 Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 6/7] generic: verify ciphertext of v1 encryption policies with AES-128 Eric Biggers 2019-04-26 20:41 ` [RFC PATCH 7/7] generic: verify ciphertext of v1 encryption policies with Adiantum Eric Biggers 2019-05-06 15:57 ` [RFC PATCH 0/7] xfstests: verify fscrypt-encrypted contents and filenames Eric Biggers 2019-05-12 12:58 ` Eryu Guan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).