Cryptsetup and hardware accelerated AES-XTS

Cryptsetup and hardware accelerated AES-XTS

[Maxim Fomin]

Hi!

Recently linux kernel got[1] faster AES-XTS on modern x86_64 CPUs thanks to VAES and AVX-10/512. I decided to dig deeper into this issue and found the article[2] from 2020 stating that dm-crypt can be configured to use faster (synchronous and hardware accelerated) algorithms with 'capi:' prefix. Can cryptsetup be configured to ask dm-crypt to use hardware accelerated algorithms?

[1] https://lore.kernel.org/lkml/20240403004404.GC2576@sol.localdomain/T/#m83293b2699f9a5da04fc5780ee402191dace3926

[2] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/

Best regards,
Maxim

Re: Cryptsetup and hardware accelerated AES-XTS

[Eric Biggers]

On Sat, Jul 20, 2024 at 12:15:23PM +0000, Maxim Fomin wrote:
> Hi!
> 
> Recently linux kernel got[1] faster AES-XTS on modern x86_64 CPUs thanks to VAES and AVX-10/512. I decided to dig deeper into this issue and found the article[2] from 2020 stating that dm-crypt can be configured to use faster (synchronous and hardware accelerated) algorithms with 'capi:' prefix. Can cryptsetup be configured to ask dm-crypt to use hardware accelerated algorithms?
> 
> [1] https://lore.kernel.org/lkml/20240403004404.GC2576@sol.localdomain/T/#m83293b2699f9a5da04fc5780ee402191dace3926
> 
> [2] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/
> 
> Best regards,
> Maxim
> 

You don't need to use "capi:".  Just make sure CONFIG_CRYPTO_AES_NI_INTEL=y is
enabled in your kernel (which it already should have been since it was needed
for AES-NI acceleration before), and the new code will be used automatically if
your CPU supports it.

- Eric

Re: Cryptsetup and hardware accelerated AES-XTS

[Maxim Fomin]

On Saturday, July 20th, 2024 at 6:36 PM, Eric Biggers <ebiggers@kernel.org> wrote:

> 
> 
> On Sat, Jul 20, 2024 at 12:15:23PM +0000, Maxim Fomin wrote:
> 
> > Hi!
> > 
> > Recently linux kernel got[1] faster AES-XTS on modern x86_64 CPUs thanks to VAES and AVX-10/512. I decided to dig deeper into this issue and found the article[2] from 2020 stating that dm-crypt can be configured to use faster (synchronous and hardware accelerated) algorithms with 'capi:' prefix. Can cryptsetup be configured to ask dm-crypt to use hardware accelerated algorithms?
> > 
> > [1] https://lore.kernel.org/lkml/20240403004404.GC2576@sol.localdomain/T/#m83293b2699f9a5da04fc5780ee402191dace3926
> > 
> > [2] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/
> > 
> > Best regards,
> > Maxim
> 
> 
> You don't need to use "capi:". Just make sure CONFIG_CRYPTO_AES_NI_INTEL=y is
> enabled in your kernel (which it already should have been since it was needed
> for AES-NI acceleration before), and the new code will be used automatically if
> your CPU supports it.
> 
> - Eric

I have heard that on some Intel CPUs AVX-512 is ignored (although it is supported) because of downclocking. I have rocketlake i5-11600K. How I can check which algorithm is actually used? How I can force cryptsetup/dm-crypt to enable AVX-512 version if it is not used?

Best regards,
Maxim

Re: Cryptsetup and hardware accelerated AES-XTS

[Eric Biggers]

On Sun, Jul 21, 2024 at 09:14:02AM +0000, Maxim Fomin wrote:
> 
> On Saturday, July 20th, 2024 at 6:36 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> 
> > 
> > 
> > On Sat, Jul 20, 2024 at 12:15:23PM +0000, Maxim Fomin wrote:
> > 
> > > Hi!
> > > 
> > > Recently linux kernel got[1] faster AES-XTS on modern x86_64 CPUs thanks to VAES and AVX-10/512. I decided to dig deeper into this issue and found the article[2] from 2020 stating that dm-crypt can be configured to use faster (synchronous and hardware accelerated) algorithms with 'capi:' prefix. Can cryptsetup be configured to ask dm-crypt to use hardware accelerated algorithms?
> > > 
> > > [1] https://lore.kernel.org/lkml/20240403004404.GC2576@sol.localdomain/T/#m83293b2699f9a5da04fc5780ee402191dace3926
> > > 
> > > [2] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/
> > > 
> > > Best regards,
> > > Maxim
> > 
> > 
> > You don't need to use "capi:". Just make sure CONFIG_CRYPTO_AES_NI_INTEL=y is
> > enabled in your kernel (which it already should have been since it was needed
> > for AES-NI acceleration before), and the new code will be used automatically if
> > your CPU supports it.
> > 
> > - Eric
> 
> I have heard that on some Intel CPUs AVX-512 is ignored (although it is
> supported) because of downclocking. I have rocketlake i5-11600K. How I can
> check which algorithm is actually used? How I can force cryptsetup/dm-crypt to
> enable AVX-512 version if it is not used?
> 

The way to tell which AES-XTS implementation is used by default on a particular
system is to check /proc/crypto for which "xts(aes)" has the highest priority.
There is a way to change algorithm priorities to override the default, but there
isn't much reason to think that people would be able to make a better choice
than the kernel's default.  The implementation that uses 512-bit vectors
(xts-aes-vaes-avx10_512) is only deprioritized on Ice Lake and Tiger Lake, so it
won't happen on your system anyway as it uses Rocket Lake.  On CPUs where the
deprioritization of xts-aes-vaes-avx10_512 does happen, there is a reason for
it, and it is only 512-bit vectors that are disabled, *not* AVX-512 entirely.

- Eric

Re: Cryptsetup and hardware accelerated AES-XTS

[Maxim Fomin]

On Sunday, July 21st, 2024 at 4:58 PM, Eric Biggers <ebiggers@kernel.org> wrote:

> 
> 
> On Sun, Jul 21, 2024 at 09:14:02AM +0000, Maxim Fomin wrote:
> 
> > On Saturday, July 20th, 2024 at 6:36 PM, Eric Biggers ebiggers@kernel.org wrote:
> > 
> > > On Sat, Jul 20, 2024 at 12:15:23PM +0000, Maxim Fomin wrote:
> > > 
> > > > Hi!
> > > > 
> > > > Recently linux kernel got[1] faster AES-XTS on modern x86_64 CPUs thanks to VAES and AVX-10/512. I decided to dig deeper into this issue and found the article[2] from 2020 stating that dm-crypt can be configured to use faster (synchronous and hardware accelerated) algorithms with 'capi:' prefix. Can cryptsetup be configured to ask dm-crypt to use hardware accelerated algorithms?
> > > > 
> > > > [1] https://lore.kernel.org/lkml/20240403004404.GC2576@sol.localdomain/T/#m83293b2699f9a5da04fc5780ee402191dace3926
> > > > 
> > > > [2] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/
> > > > 
> > > > Best regards,
> > > > Maxim
> > > 
> > > You don't need to use "capi:". Just make sure CONFIG_CRYPTO_AES_NI_INTEL=y is
> > > enabled in your kernel (which it already should have been since it was needed
> > > for AES-NI acceleration before), and the new code will be used automatically if
> > > your CPU supports it.
> > > 
> > > - Eric
> > 
> > I have heard that on some Intel CPUs AVX-512 is ignored (although it is
> > supported) because of downclocking. I have rocketlake i5-11600K. How I can
> > check which algorithm is actually used? How I can force cryptsetup/dm-crypt to
> > enable AVX-512 version if it is not used?
> 
> 
> The way to tell which AES-XTS implementation is used by default on a particular
> system is to check /proc/crypto for which "xts(aes)" has the highest priority.

Does the priority property takes into account deprioritization of Ice Lake and Tiger Lake?
Btw, actual function can be noticed in 'perf' command output, which in my case confirms that indeed vaes-avx10-512 version is used.

> There is a way to change algorithm priorities to override the default, but there
> isn't much reason to think that people would be able to make a better choice
> than the kernel's default.

Default choice may be problematic because it is applied in every case and does not take into account option to disable downclocking and thermal limit on actual machine. Some BIOS versions allow to disable downclocking (called AVX512 ratio offset or similar). If this option is available and thermal limit is not applicable, then default kernel policy means some potential CPU frequency is lost. 

> The implementation that uses 512-bit vectors
> (xts-aes-vaes-avx10_512) is only deprioritized on Ice Lake and Tiger Lake, so it
> won't happen on your system anyway as it uses Rocket Lake. On CPUs where the
> deprioritization of xts-aes-vaes-avx10_512 does happen, there is a reason for
> it, and it is only 512-bit vectors that are disabled, not AVX-512 entirely.
> 
> - Eric

Thanks for clarification. Just for reference - this setting currently resides at arch/x86/crypto/aesni-intel_glue.c and can be moved somewhere else as is written in the patch.

Best regards,
Maxim

Re: Cryptsetup and hardware accelerated AES-XTS

[Eric Biggers]

On Mon, Jul 22, 2024 at 10:36:59AM +0000, Maxim Fomin wrote:
> 
> On Sunday, July 21st, 2024 at 4:58 PM, Eric Biggers <ebiggers@kernel.org> wrote:
> 
> > 
> > 
> > On Sun, Jul 21, 2024 at 09:14:02AM +0000, Maxim Fomin wrote:
> > 
> > > On Saturday, July 20th, 2024 at 6:36 PM, Eric Biggers ebiggers@kernel.org wrote:
> > > 
> > > > On Sat, Jul 20, 2024 at 12:15:23PM +0000, Maxim Fomin wrote:
> > > > 
> > > > > Hi!
> > > > > 
> > > > > Recently linux kernel got[1] faster AES-XTS on modern x86_64 CPUs thanks to VAES and AVX-10/512. I decided to dig deeper into this issue and found the article[2] from 2020 stating that dm-crypt can be configured to use faster (synchronous and hardware accelerated) algorithms with 'capi:' prefix. Can cryptsetup be configured to ask dm-crypt to use hardware accelerated algorithms?
> > > > > 
> > > > > [1] https://lore.kernel.org/lkml/20240403004404.GC2576@sol.localdomain/T/#m83293b2699f9a5da04fc5780ee402191dace3926
> > > > > 
> > > > > [2] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/
> > > > > 
> > > > > Best regards,
> > > > > Maxim
> > > > 
> > > > You don't need to use "capi:". Just make sure CONFIG_CRYPTO_AES_NI_INTEL=y is
> > > > enabled in your kernel (which it already should have been since it was needed
> > > > for AES-NI acceleration before), and the new code will be used automatically if
> > > > your CPU supports it.
> > > > 
> > > > - Eric
> > > 
> > > I have heard that on some Intel CPUs AVX-512 is ignored (although it is
> > > supported) because of downclocking. I have rocketlake i5-11600K. How I can
> > > check which algorithm is actually used? How I can force cryptsetup/dm-crypt to
> > > enable AVX-512 version if it is not used?
> > 
> > 
> > The way to tell which AES-XTS implementation is used by default on a particular
> > system is to check /proc/crypto for which "xts(aes)" has the highest priority.
> 
> Does the priority property takes into account deprioritization of Ice Lake and
> Tiger Lake?

Yes.

> Btw, actual function can be noticed in 'perf' command output, which in my case
> confirms that indeed vaes-avx10-512 version is used.
> 
> > There is a way to change algorithm priorities to override the default, but there
> > isn't much reason to think that people would be able to make a better choice
> > than the kernel's default.
> 
> Default choice may be problematic because it is applied in every case and does
> not take into account option to disable downclocking and thermal limit on
> actual machine. Some BIOS versions allow to disable downclocking (called
> AVX512 ratio offset or similar).

Sure, but people who disable downclocking may not have fully tested their system
for stability when executing workloads that use 512-bit vectors extensively.
There's a very real risk of the system crashing during such workloads if the CPU
is using unsupported operating parameters.

> If this option is available and thermal limit is not applicable, then default
> kernel policy means some potential CPU frequency is lost. 

Note that in AES-XTS microbenchmarks on Ice Lake, there's a ~127% improvement
from moving from the old implementation to xts-aes-vaes-avx10_256, but only a
~11% further improvement from moving to xts-aes-vaes-avx10_512.  I.e., 512-bit
vectors provide little benefit on that CPU in the first place for AES-XTS...  So
why take on the risks of using them?

- Eric

Re: Cryptsetup and hardware accelerated AES-XTS

[Milan Broz]

On 7/20/24 2:15 PM, Maxim Fomin wrote:
> Recently linux kernel got[1] faster AES-XTS on modern x86_64 CPUs
> thanks to VAES and AVX-10/512. I decided to dig deeper into this
> issue and found the article[2] from 2020 stating that dm-crypt can be
> configured to use faster (synchronous and hardware accelerated)
> algorithms with 'capi:' prefix. Can cryptsetup be configured to ask
> dm-crypt to use hardware accelerated algorithms?

Hi,

TL;DR - you cannot use "capi" driver specification with cryptsetup.

As Eric said in other reply, it is up to the kernel to set priority
which driver is used in kernel. You can always blacklist some modules.


For the archive, some context below.

The "capi" format support in dm-crypt was introduced due to the support
for authenticated encryption where original cipher dm-crypt format cannot
be used.
It was actually my mistake that we allowed specification of specific
*driver* in dm-crypt mapping table - this is really not job for dm-crypt.

Currently, while dm-crypt allows that driver specification, cryptsetup
will reject it as it is possible security problem.
Quoting from the cryptsetup 2.7.0 release notes
https://www.kernel.org/pub/linux/utils/cryptsetup/v2.7/v2.7.0-ReleaseNotes


* Disallow the use of internal kernel crypto driver names in "capi"
   specification.

   The common way to specify cipher mode in cryptsetup is to use
   cipher-mode-iv notation (like aes-xts-plain64).
   With the introduction of authenticated ciphers, we also allow
   "capi:<spec>" notation that is directly used by dm-crypt
   (e.g., capi:xts(aes)-plain64).

   CAPI specification was never intended to be used directly in the LUKS
   header; unfortunately, the code allowed it until now.
   Devices with CAPI specification in metadata can no longer be activated;
   header repair is required.

   CAPI specification could allow attackers to change the cipher
   specification to enforce loading some specific kernel crypto driver
   (for example, load driver with known side-channel issues).
   This can be problematic, specifically in a cloud environment
   (modifying LUKS2 metadata in container image).

   Thanks to Jan Wichelmann, Luca Wilke, and Thomas Eisenbarth from
   University of Luebeck for noticing the problems with this code.

Thanks,
Milan