[dm-crypt] Streaming ciphers

[dm-crypt] Streaming ciphers

[David Niklas]

Hello,
Today I was helping a poor luser who's Crucial P1 QLC SSD died from
too many writes. We still don't know what's doing the writing, but he is
running luks using FDE so write amplification is definitely a
factor.
I forget the answer to this so please enlighten me, is there a streaming
cypher he can use instead of a block cipher? A better option that I
didn't consider?
Not that the TCG Opal is to be fully trusted (nor is it available on
that model), but how does it work with respect to write amplification vs.
luks?

Thanks,
David

Re: [dm-crypt] Streaming ciphers

[Arno Wagner]

Hi David,

using LUKS should not result in write-amplification.
LUKS uses 512B blocks and just provides an 1:1 mapping.

Regards,
Arno
 

On Sun, Feb 16, 2020 at 01:59:41 CET, David Niklas wrote:
> Hello,
> Today I was helping a poor luser who's Crucial P1 QLC SSD died from
> too many writes. We still don't know what's doing the writing, but he is
> running luks using FDE so write amplification is definitely a
> factor.
> I forget the answer to this so please enlighten me, is there a streaming
> cypher he can use instead of a block cipher? A better option that I
> didn't consider?
> Not that the TCG Opal is to be fully trusted (nor is it available on
> that model), but how does it work with respect to write amplification vs.
> luks?
> 
> Thanks,
> David
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Streaming ciphers

[Matthias Schniedermeyer]

On 16.02.2020 03:32, Arno Wagner wrote:
> Hi David,
> 
> using LUKS should not result in write-amplification.
> LUKS uses 512B blocks and just provides an 1:1 mapping.

Using defaults it does cause a higher write-amplification.
By default dm-crypt doesn't not enable discard-support, whereas it is 
normaly supported for an unencrypted filesystem.

And given that at least some distributions even install a 
fstrim-cronjob, the filesystems are trimmed, even if not mounted with 
the discard-option.

This results in an unencrypted filesystem having it's empty space 
"empty", as far as the SSD is concerned.
Wheras "empty" space (either "is" or "stays") "used" for a LUKS device.

The "is" or "stays" difference comes with having the encrypted device 
pre-inited or not. With pre-init the blocks are always "in use". Without 
pre-init the blocks are "unused" as long as they aren't written and 
switch to "in use" after first beeing written, after which they will 
stay "in use" forever.

Less "empty" space results in higher write-amplification because the SSD 
has to remap "empty" space as it is "in use", as far as the SSD is 
concerned.

A pre-inited encrypted device, in this scenario, never has any empty 
space.


I'm not saying that you can't make it equivalent, just that there are 
differences when using the defaults.


> On Sun, Feb 16, 2020 at 01:59:41 CET, David Niklas wrote:
> > Hello,
> > Today I was helping a poor luser who's Crucial P1 QLC SSD died from
> > too many writes. We still don't know what's doing the writing, but he is
> > running luks using FDE so write amplification is definitely a
> > factor.
> > I forget the answer to this so please enlighten me, is there a streaming
> > cypher he can use instead of a block cipher? A better option that I
> > didn't consider?
> > Not that the TCG Opal is to be fully trusted (nor is it available on
> > that model), but how does it work with respect to write amplification vs.
> > luks?
> > 
> > Thanks,
> > David
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > https://www.saout.de/mailman/listinfo/dm-crypt
> 
> -- 
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. -- Plato
> 
> If it's in the news, don't worry about it.  The very definition of 
> "news" is "something that hardly ever happens." -- Bruce Schneier
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt

-- 

Matthias

Re: [dm-crypt] Streaming ciphers

[David Niklas]

Yuck, top post. Reformat and reply.

On Sun, 16 Feb 2020 14:58:10 +0100
Matthias Schniedermeyer <ms@citd.de> wrote:
> > On Sun, Feb 16, 2020 at 01:59:41 CET, David Niklas wrote:
> > > Hello,
> > > Today I was helping a poor luser who's Crucial P1 QLC SSD died from
> > > too many writes. We still don't know what's doing the writing, but
> > > he is running luks using FDE so write amplification is definitely a
> > > factor.
> > > I forget the answer to this so please enlighten me, is there a
> > > streaming cypher he can use instead of a block cipher? A better
> > > option that I didn't consider?
> > > Not that the TCG Opal is to be fully trusted (nor is it available on
> > > that model), but how does it work with respect to write
> > > amplification vs. luks?
> > >
> > > Thanks,
> > > David
> On 16.02.2020 03:32, Arno Wagner wrote:
> > Hi David,
> >
> > using LUKS should not result in write-amplification.
> > LUKS uses 512B blocks and just provides an 1:1 mapping.
>
> Using defaults it does cause a higher write-amplification.
> By default dm-crypt doesn't not enable discard-support, whereas it is
> normaly supported for an unencrypted filesystem.
>
> And given that at least some distributions even install a
> fstrim-cronjob, the filesystems are trimmed, even if not mounted with
> the discard-option.
>
> This results in an unencrypted filesystem having it's empty space
> "empty", as far as the SSD is concerned.
> Wheras "empty" space (either "is" or "stays") "used" for a LUKS device.
>
> The "is" or "stays" difference comes with having the encrypted device
> pre-inited or not. With pre-init the blocks are always "in use".
> Without pre-init the blocks are "unused" as long as they aren't written
> and switch to "in use" after first beeing written, after which they
> will stay "in use" forever.
>
> Less "empty" space results in higher write-amplification because the
> SSD has to remap "empty" space as it is "in use", as far as the SSD is
> concerned.
>
> A pre-inited encrypted device, in this scenario, never has any empty
> space.
>
>
> I'm not saying that you can't make it equivalent, just that there are
> differences when using the defaults.

Thanks! I understand.
Is there anything else I should be aware of (aside from a kernel option I
need to look up again to delay writing to disk), that will lessen disk
writes/amplification from luks?

Thanks,
David

Re: [dm-crypt] Streaming ciphers

[Matthias Schniedermeyer]

On 16.02.2020 22:56, David Niklas wrote:
> Yuck, top post. Reformat and reply.
> 
> On Sun, 16 Feb 2020 14:58:10 +0100
> Matthias Schniedermeyer <ms@citd.de> wrote:
> > > On Sun, Feb 16, 2020 at 01:59:41 CET, David Niklas wrote:
> > > > Hello,
> > > > Today I was helping a poor luser who's Crucial P1 QLC SSD died from
> > > > too many writes. We still don't know what's doing the writing, but
> > > > he is running luks using FDE so write amplification is definitely a
> > > > factor.
> > > > I forget the answer to this so please enlighten me, is there a
> > > > streaming cypher he can use instead of a block cipher? A better
> > > > option that I didn't consider?
> > > > Not that the TCG Opal is to be fully trusted (nor is it available on
> > > > that model), but how does it work with respect to write
> > > > amplification vs. luks?
> > > >
> > > > Thanks,
> > > > David
> > On 16.02.2020 03:32, Arno Wagner wrote:
> > > Hi David,
> > >
> > > using LUKS should not result in write-amplification.
> > > LUKS uses 512B blocks and just provides an 1:1 mapping.
> >
> > Using defaults it does cause a higher write-amplification.
> > By default dm-crypt doesn't not enable discard-support, whereas it is
> > normaly supported for an unencrypted filesystem.
> >
> > And given that at least some distributions even install a
> > fstrim-cronjob, the filesystems are trimmed, even if not mounted with
> > the discard-option.
> >
> > This results in an unencrypted filesystem having it's empty space
> > "empty", as far as the SSD is concerned.
> > Wheras "empty" space (either "is" or "stays") "used" for a LUKS device.
> >
> > The "is" or "stays" difference comes with having the encrypted device
> > pre-inited or not. With pre-init the blocks are always "in use".
> > Without pre-init the blocks are "unused" as long as they aren't written
> > and switch to "in use" after first beeing written, after which they
> > will stay "in use" forever.
> >
> > Less "empty" space results in higher write-amplification because the
> > SSD has to remap "empty" space as it is "in use", as far as the SSD is
> > concerned.
> >
> > A pre-inited encrypted device, in this scenario, never has any empty
> > space.
> >
> >
> > I'm not saying that you can't make it equivalent, just that there are
> > differences when using the defaults.
> 
> Thanks! I understand.
> Is there anything else I should be aware of (aside from a kernel option I
> need to look up again to delay writing to disk), that will lessen disk
> writes/amplification from luks?

If you don't have a "SSD killing write load" (at least hundreth's of GB/s 
per day) you normaly don't have to worry about killing any halfway recent SSD.

Of course you can always have bad luck. I personally had 2 enterprise 
SSDs die on me @work, both after only a few weeks in production.
(IOW "Infant Death" category)

But i've no personal experience with "very recent" QLC drives.
QLC drives are expected to have an even lower Write Endurance than TLCs.

Given that the Crucial P1 appears to have entered the market just over a 
years ago it is a quite recent drive.

The 1TB drive is listed with an endurance of 200TB and 5 year warranty.
That makes about 110GB per day, or about 0.11 DWPD.

So i would guess that meantioned drive is less than 1 year old.

Unless you had written to the SSD in excess of 600GB per day continously 
since it was bought, it was likely more because of "bad luck" that the 
SSD died.




-- 

Matthias