Filling a device with random data

Filling a device with random data

[David Joaquín Shourabi Porcel]

Hey there :)

Section 2.19 of the [FAQ][1] explains how to fill devices with random data using dm-crypt: cryptsetup opens a plain mapping, zeros are written to it and it is closed. If LUKS is set up on the device afterwards, this translates to the following commands:

```
cryptsetup open --type=plain --key-file=/dev/urandom -- /dev/foo temporary
dd if=/dev/zero of=/dev/mapper/temporary
cryptsetup close -- temporary
cryptsetup luksFormat -- /dev/foo
cryptsetup open -- /dev/foo bar
# mkfs / pvcreate -- /dev/mapper/bar
```

I would like to skip the first mapping and shorten the procedure as follows:

```
cryptsetup luksFormat -- /dev/foo
cryptsetup open -- /dev/foo bar
dd if=/dev/zero of=/dev/mapper/bar
# mkfs / pvcreate -- /dev/mapper/bar
```

Does filling a device with random data under a separate dm-crypt mapping offer any security benefit?

Kind regards,

David J. Shourabi Porcel


[1]: https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions

Re: Filling a device with random data

[Arno Wagner]

Hi,

/dev/urandom has gotten a lot faster. You should be able to
write the data from it directly to the device without  
the plain dm-crypt mapping in between and still get decent
speed.

That said, you can also zero-overwrite a mapped LUKS
container for pretty much the same level of security.
The method with the plain dm-crypt mapping in between 
is _old_. 

Regards,
Arno

On Wed, Feb 08, 2023 at 22:09:54 CET, David Joaquín Shourabi Porcel wrote:
> Hey there :)
> 
> Section 2.19 of the [FAQ][1] explains how to fill devices with random data using dm-crypt: cryptsetup opens a plain mapping, zeros are written to it and it is closed. If LUKS is set up on the device afterwards, this translates to the following commands:
> 
> ```
> cryptsetup open --type=plain --key-file=/dev/urandom -- /dev/foo temporary
> dd if=/dev/zero of=/dev/mapper/temporary
> cryptsetup close -- temporary
> cryptsetup luksFormat -- /dev/foo
> cryptsetup open -- /dev/foo bar
> # mkfs / pvcreate -- /dev/mapper/bar
> ```
> 
> I would like to skip the first mapping and shorten the procedure as follows:
> 
> ```
> cryptsetup luksFormat -- /dev/foo
> cryptsetup open -- /dev/foo bar
> dd if=/dev/zero of=/dev/mapper/bar
> # mkfs / pvcreate -- /dev/mapper/bar
> ```
> 
> Does filling a device with random data under a separate dm-crypt mapping
> offer any security benefit?
> 
> Kind regards,
> 
> David J. Shourabi Porcel
> 
> 
> [1]: https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: Filling a device with random data

[David Joaquín Shourabi Porcel]

Thank you for your answer, Arno :). I benchmarked both methods and share my results in case they are of use to someone.

On my system, dm-crypt in its default configuration (AES in XTS mode with two 256 bit keys) is about ten times as fast as /dev/urandom:

```
$ dd if=/dev/urandom of=/dev/null status=progress
4819285504 bytes (4,8 GB, 4,5 GiB) copied, 12 s, 402 MB/s^C
9708379+0 records in
9708378+0 records out
4970689536 bytes (5,0 GB, 4,6 GiB) copied, 12,379 s, 402 MB/s

$ cryptsetup benchmark --cipher=aes-xts-plain64 --key-size=512
# Tests are approximate using memory only (no storage IO).
# Algorithm |       Key |      Encryption |      Decryption
    aes-xts        512b      4749,3 MiB/s      4622,4 MiB/s
```

/dev/urandom also provides data at a lower rate when multiple processes read from it simultaneously; dm-crypt did not seem to slow down. This might be worth considering if several drives or partitions are to be filled with random data in parallel. On the other hand, /dev/urandom is much faster than any of the devices I will be preparing for encryption.

Kind regards,

David J. Shourabi Porcel

Re: Filling a device with random data

[Michael Kjörling]

On 9 Feb 2023 16:01 +0100, from david@djsp.eu (David Joaquín Shourabi Porcel):
> $ dd if=/dev/urandom of=/dev/null status=progress
> 4819285504 bytes (4,8 GB, 4,5 GiB) copied, 12 s, 402 MB/s^C
> 9708379+0 records in
> 9708378+0 records out
> 4970689536 bytes (5,0 GB, 4,6 GiB) copied, 12,379 s, 402 MB/s

That's doing I/O in blocks of 512 bytes, so syscall overhead is likely
to be a large limiting factor. (Each block requires at least a few
kernel/userspace context switches through syscalls; at an _absolute_
minimum, for each block, it's likely going to go through userspace dd
to kernel read through kernel /dev/urandom driver to userspace dd to
kernel write through kernel /dev/null driver to userspace dd.) Compare
and contrast that to a cryptsetup benchmark which is likely (I haven't
looked) primarily one or the other. If you want to gauge the
performance of /dev/urandom, I suggest trying with something like
bs=1M to reduce that overhead. Going from bs=512 to bs=1M I get an
about 60% speed increase, although it still comes up far short of a
cryptsetup benchmark (with bs=1M at about a quarter of the throughput
of the cryptsetup benchmark on my particular system). So for fast
storage media, going through a crypt mapping can still be beneficial,
although the degree of benefit will obviously depend on the relative
performance of CPU (including encryption instructions) and storage
device. Whether you use a plain dm-crypt mapping or a LUKS container
should be largely irrelevant for these purposes, however.

Similarly, when you're writing to a real-life storage device, doing so
in blocks of 512 bytes is going to utterly devastate performance
compared to using blocks on the order of tens of megabytes or even
larger. (I realize that's not what either of your tests do.)

Also, what kernel version are you running?

The major advantage to doing a separate full-disk write with a
throwaway key is that the data is going to be meaningless also through
the eyes of the final LUKS container mapping. Depending on your use
case, this can provide a slight confidentiality advantage.

Doing a full-disk overwrite pass on SSDs has its problems, though;
primarily that it forces the SSD to consider every user-accessible
block as being in use, hampering wear leveling. That becomes a far
more delicate balancing act between storage metadata confidentiality
(the full-disk overwrite hampers an attacker's ability to determine
what blocks are in use) and giving the SSD's firmware a decent chance
of doing wear leveling (for which you want to leave the underlying
storage pristine and perhaps also allow TRIM passthrough, but which
leaks information about which parts of the storage device are in use).

-- 
Michael Kjörling                     🔗 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

Re: Filling a device with random data

[Eric Biggers]

On Thu, Feb 09, 2023 at 03:37:56PM +0000, Michael Kjörling wrote:
> On 9 Feb 2023 16:01 +0100, from david@djsp.eu (David Joaquín Shourabi Porcel):
> > $ dd if=/dev/urandom of=/dev/null status=progress
> > 4819285504 bytes (4,8 GB, 4,5 GiB) copied, 12 s, 402 MB/s^C
> > 9708379+0 records in
> > 9708378+0 records out
> > 4970689536 bytes (5,0 GB, 4,6 GiB) copied, 12,379 s, 402 MB/s
> 
> That's doing I/O in blocks of 512 bytes, so syscall overhead is likely
> to be a large limiting factor. (Each block requires at least a few
> kernel/userspace context switches through syscalls; at an _absolute_
> minimum, for each block, it's likely going to go through userspace dd
> to kernel read through kernel /dev/urandom driver to userspace dd to
> kernel write through kernel /dev/null driver to userspace dd.) Compare
> and contrast that to a cryptsetup benchmark which is likely (I haven't
> looked) primarily one or the other. If you want to gauge the
> performance of /dev/urandom, I suggest trying with something like
> bs=1M to reduce that overhead. Going from bs=512 to bs=1M I get an
> about 60% speed increase, although it still comes up far short of a
> cryptsetup benchmark (with bs=1M at about a quarter of the throughput
> of the cryptsetup benchmark on my particular system). So for fast
> storage media, going through a crypt mapping can still be beneficial,
> although the degree of benefit will obviously depend on the relative
> performance of CPU (including encryption instructions) and storage
> device. Whether you use a plain dm-crypt mapping or a LUKS container
> should be largely irrelevant for these purposes, however.
> 
> Similarly, when you're writing to a real-life storage device, doing so
> in blocks of 512 bytes is going to utterly devastate performance
> compared to using blocks on the order of tens of megabytes or even
> larger. (I realize that's not what either of your tests do.)
> 
> Also, what kernel version are you running?
> 
> The major advantage to doing a separate full-disk write with a
> throwaway key is that the data is going to be meaningless also through
> the eyes of the final LUKS container mapping. Depending on your use
> case, this can provide a slight confidentiality advantage.
> 
> Doing a full-disk overwrite pass on SSDs has its problems, though;
> primarily that it forces the SSD to consider every user-accessible
> block as being in use, hampering wear leveling. That becomes a far
> more delicate balancing act between storage metadata confidentiality
> (the full-disk overwrite hampers an attacker's ability to determine
> what blocks are in use) and giving the SSD's firmware a decent chance
> of doing wear leveling (for which you want to leave the underlying
> storage pristine and perhaps also allow TRIM passthrough, but which
> leaks information about which parts of the storage device are in use).
> 

While this is a creative use of dm-crypt, a cleaner way to generate very high
throughput cryptographically secure random data (specifically, even higher
throughput than /dev/urandom which is usually good enough) is to just generate
it in userspace.  For example, the command 'openssl rand $LENGTH' does this.

- Eric

Re: Filling a device with random data

[Arno Wagner]

As I said, it is old. Old like some not having 
hardware support and /dev/urandom being dog-slow.

Toaday, it is not really a problem doing this fast.

Regards,
Arno



On Thu, Feb 09, 2023 at 20:03:52 CET, Eric Biggers wrote:
> While this is a creative use of dm-crypt, a cleaner way to generate very
> high throughput cryptographically secure random data (specifically, even
> higher throughput than /dev/urandom which is usually good enough) is to
> just generate it in userspace.  For example, the command 'openssl rand
> $LENGTH' does this.
> 
> - Eric

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: Filling a device with random data

[Arno Wagner]

I should add that I though about removing it, but 
this approach may still have a use on very small 
Linux devices. Meube I should add a comment to that
effect.

Arno

On Tue, Feb 14, 2023 at 23:17:11 CET, Arno Wagner wrote:
> As I said, it is old. Old like some not having 
> hardware support and /dev/urandom being dog-slow.
> 
> Toaday, it is not really a problem doing this fast.
> 
> Regards,
> Arno
> 
> 
> 
> On Thu, Feb 09, 2023 at 20:03:52 CET, Eric Biggers wrote:
> > While this is a creative use of dm-crypt, a cleaner way to generate very
> > high throughput cryptographically secure random data (specifically, even
> > higher throughput than /dev/urandom which is usually good enough) is to
> > just generate it in userspace.  For example, the command 'openssl rand
> > $LENGTH' does this.
> > 
> > - Eric
> 
> -- 
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. -- Plato
> 
> If it's in the news, don't worry about it.  The very definition of 
> "news" is "something that hardly ever happens." -- Bruce Schneier

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier