Encryption implementation like ZFS?

Encryption implementation like ZFS?

[Sandra Schlichting]

Hi everybody,

According to [0] ZFS does encryption:

One exception to this is the encryption support being added to the ZFS
filesystem. Filesystem metadata such as filenames, ownership, ACLs,
extended attributes are all stored encrypted on disk. The ZFS metadata
about the storage pool is still stored in the clear so it is possible
to determine how many filesystems (datasets) are available in the pool
and even which ones are encrypted but not what the content of the
stored files or directories are.

Will btrfs implement encryption the same way?

I read that the XFS encryption work have stalled. Does this impact btrfs?

Best regards,
Sandra

[0] http://en.wikipedia.org/wiki/Filesystem-level_encryption

Re: Encryption implementation like ZFS?

[Billy Crook]

On Fri, Dec 30, 2011 at 12:32, Sandra Schlichting
<littlesandra88@gmail.com> wrote:
> According to [0] ZFS does encryption:
>
> One exception to this is the encryption support being added to the ZFS
> filesystem. Filesystem metadata such as filenames, ownership, ACLs,
> extended attributes are all stored encrypted on disk. The ZFS metadata
> about the storage pool is still stored in the clear so it is possible
> to determine how many filesystems (datasets) are available in the pool
> and even which ones are encrypted but not what the content of the
> stored files or directories are.

How is this advantageous over dmcrypt-LUKS?

LUKS has always been supported between the block device and a btrfs
filesystem, and would not even let an attacker discern what type of
file-system was in use.

Re: Encryption implementation like ZFS?

[Sandra Schlichting]

> How is this advantageous over dmcrypt-LUKS?

TRIM pass-through for SSD's. With dmcrypt on an SSD write performance
is very slow.

> LUKS has always been supported between the block device and a btrfs
> filesystem, and would not even let an attacker discern what type of
> file-system was in use.

cryptsetup-1.4.0-1 have --enable-discards option to allow
discards/TRIM, but it is not recommended for theoretical security
reasons.

>From my understanding some btrfs features wouldn't work with dmcrypt,
like "online volume growth and shrinking"?

Re: Encryption implementation like ZFS?

[Billy Crook]

On Fri, Dec 30, 2011 at 14:12, Sandra Schlichting
<littlesandra88@gmail.com> wrote:
>> How is this advantageous over dmcrypt-LUKS?
>
> TRIM pass-through for SSD's. With dmcrypt on an SSD write performance
> is very slow.

Good point.  I'm actually very close to moving from magnetic to SSD
storage for my btrfs volumes.  Would my luks layer offset the majority
of any advantage I might otherwise see from SSD?  I'd be happy just to
eliminate seektime.

> cryptsetup-1.4.0-1 have --enable-discards option to allow
> discards/TRIM, but it is not recommended for theoretical security
> reasons.
>
> From my understanding some btrfs features wouldn't work with dmcrypt,
> like "online volume growth and shrinking"?

I've been using btrfs on luks now for 6 months and online grow works
fine -- if you online grow the luks container first. (cryptsetup
resize <name>)  It does make it a more manual process, I suppose, but
for me it's worth knowing that no part of the filesystem, will ever be
available should a disk be stolen.

Re: Encryption implementation like ZFS?

[Sandra Schlichting]

> Good point. =A0I'm actually very close to moving from magnetic to SSD
> storage for my btrfs volumes. =A0Would my luks layer offset the major=
ity
> of any advantage I might otherwise see from SSD? =A0I'd be happy just=
 to
> eliminate seektime.

Not to my knowledge.

> I've been using btrfs on luks now for 6 months and online grow works
> fine -- if you online grow the luks container first. (cryptsetup
> resize <name>) =A0It does make it a more manual process, I suppose, b=
ut
> for me it's worth knowing that no part of the filesystem, will ever b=
e
> available should a disk be stolen.

That is of course true =3D)

That is sort the EXT2/3/4 way of doing it, where it would be very
handy if btrfs adapted the ZFS idea, where the FS knows and does
everything, where everything is done through the btrfs commands.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Encryption implementation like ZFS?

[Tomasz Torcz]

On Fri, Dec 30, 2011 at 01:27:43PM -0600, Billy Crook wrote:
> On Fri, Dec 30, 2011 at 12:32, Sandra Schlichting
> <littlesandra88@gmail.com> wrote:
> > According to [0] ZFS does encryption:
> >
> > One exception to this is the encryption support being added to the ZFS
> > filesystem. Filesystem metadata such as filenames, ownership, ACLs,
> > extended attributes are all stored encrypted on disk. The ZFS metadata
> > about the storage pool is still stored in the clear so it is possible
> > to determine how many filesystems (datasets) are available in the pool
> > and even which ones are encrypted but not what the content of the
> > stored files or directories are.
> 
> How is this advantageous over dmcrypt-LUKS?

  For example mixing encrypted and not encrypted subvolumes in one pool.
And not having to separately cryptsetup luksOpen all disks consisting filesystem.
  There are advantages of FDE like dm-crypt and selective encryption like in ZFS.

-- 
Tomasz Torcz               RIP is irrevelant. Spoofing is futile.
xmpp: zdzichubg@chrome.pl     Your routes will be aggreggated. -- Alex Yuriev

Re: Encryption implementation like ZFS?

[Niels de Carpentier]

>> How is this advantageous over dmcrypt-LUKS?
>
>   For example mixing encrypted and not encrypted subvolumes in one pool.
> And not having to separately cryptsetup luksOpen all disks consisting
> filesystem.
>   There are advantages of FDE like dm-crypt and selective encryption like
> in ZFS.

I think there were possible problems with btrfs on dm-crypt before, which
was suspected to due to write barrier issues of dm-crypt. Has this been
fixed?

Niels

Re: Encryption implementation like ZFS?

[Fajar A. Nugraha]

On Sat, Dec 31, 2011 at 3:12 AM, Sandra Schlichting
<littlesandra88@gmail.com> wrote:
>> How is this advantageous over dmcrypt-LUKS?
>
> TRIM pass-through for SSD's. With dmcrypt on an SSD write performance
> is very slow.

... and depending on which SSD you use, it shouldn't matter. Really.

Last time I tried with sandforce SSD + btrfs + -o discard, forcing
trim actually made things slower. Sandforce (and probably other modern
SSD) controllers can work just fine even without explicit trim fs
support.

-- 
Fajar

Re: Encryption implementation like ZFS?

[Felix Blanke]

 Hi,
  
I tried my SSD with luks and it is damn slow. I don't have any numbers for i/o per 
second but the read/write speed was <50% of the speed without luks. If you want 
encryption and speed you have to use loopaes. Unfortunetly you have to patch your
util-linux for that. 
The advantage of loopaes is that it is more secure then luks and way faster.
The speed with encryption (loopaes) is about 90-95% the speed without encryption at
my setup.
           
Best regards,
Felix 


On 30. December 2011 - 14:49, Billy Crook wrote:
> Date:	Fri, 30 Dec 2011 14:49:06 -0600
> From: Billy Crook <billycrook@gmail.com>
> To: Sandra Schlichting <littlesandra88@gmail.com>
> Cc: linux-btrfs@vger.kernel.org
> Subject: Re: Encryption implementation like ZFS?
> 
> On Fri, Dec 30, 2011 at 14:12, Sandra Schlichting
> <littlesandra88@gmail.com> wrote:
> >> How is this advantageous over dmcrypt-LUKS?
> >
> > TRIM pass-through for SSD's. With dmcrypt on an SSD write performance
> > is very slow.
> 
> Good point.  I'm actually very close to moving from magnetic to SSD
> storage for my btrfs volumes.  Would my luks layer offset the majority
> of any advantage I might otherwise see from SSD?  I'd be happy just to
> eliminate seektime.
> 
> > cryptsetup-1.4.0-1 have --enable-discards option to allow
> > discards/TRIM, but it is not recommended for theoretical security
> > reasons.
> >
> > From my understanding some btrfs features wouldn't work with dmcrypt,
> > like "online volume growth and shrinking"?
> 
> I've been using btrfs on luks now for 6 months and online grow works
> fine -- if you online grow the luks container first. (cryptsetup
> resize <name>)  It does make it a more manual process, I suppose, but
> for me it's worth knowing that no part of the filesystem, will ever be
> available should a disk be stolen.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
---end quoted text---

Re: Encryption implementation like ZFS?

[Sandra Schlichting]

> ... and depending on which SSD you use, it shouldn't matter. Really.
>
> Last time I tried with sandforce SSD + btrfs + -o discard, forcing
> trim actually made things slower. Sandforce (and probably other modern
> SSD) controllers can work just fine even without explicit trim fs
> support.

What command did you use to test this?

I have an OCZ Agility 3 SSD, which have the latest Sandforce
controller, so I would really like to try reproduce your test setup.

Re: Encryption implementation like ZFS?

[Sandra Schlichting]

> The advantage of loopaes is that it is more secure then luks and way faster.
> The speed with encryption (loopaes) is about 90-95% the speed without encryption at
> my setup.

Sounds very interesting. Will try that.

I case of my PC I would actually prefer a weaker encryption, as I am
not protecting my data from NSA, but from a person with a non-crypto
background, but I don't suppose btrfs will have such option =)

In terms of parallelization, would there be benefits of having the
encryption done by btrfs?

RE: Encryption implementation like ZFS?

[Edward Ned Harvey]

> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-
> owner@vger.kernel.org] On Behalf Of Sandra Schlichting
> 
> TRIM pass-through for SSD's. With dmcrypt on an SSD write performance
> is very slow.

That's a relative term.  Suppose the SSD does 60x higher random IOPS than
the HDD, in an optimal configuration.  Without support for TRIM, it will
degrade something like 2x to 4x, so you will only get 15x to 30x higher
random write IOPS than the HDD.  Remember also, that the whole TRIM topic is
only applicable to writes.  Regardless of TRIM or encryption, your random
read IOPS are going to be far higher with the SSD.  The full 60x random read
IOPS regardless of TRIM.

But random IOPS is only one measure... If the SSD and the HDD are about the
same sequential throughput in the optimal configuration, then the SSD
degrades down to 1/2  to 1/4 the speed of the HDD for sequential writes in
the absence of TRIM.  This is a very sticky subject - because what the mfgr
publishes for specs on the SSD are usually the out-of-the-box performance
specs, under ideal conditions.  In reality, it's rarely even on the same
order as the real life measured performance after your system has been in
use for a few weeks.  YMMV dramatically.

Quickly looking at commodity SSD's on the market right now, I see claims of
130 to 200 MB/sec sustained.  This compares to HDD's which are right around
1.0Gbit/sec = 125 MB/sec.  In my personal experience, I would say the 130 to
200 MB is only true for a brand new SSD in optimal configuration.  I would
expect these drives in real life usage to be very comparable to the HDD for
sequential reads, and writes if TRIM is supported, but the SSD significantly
slower than the HDD for sequential writes without TRIM.


> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-
> owner@vger.kernel.org] On Behalf Of Billy Crook
> 
> Good point.  I'm actually very close to moving from magnetic to SSD
> storage for my btrfs volumes.  Would my luks layer offset the majority
> of any advantage I might otherwise see from SSD?  I'd be happy just to
> eliminate seektime.

If you mostly do random reads, random writes, or sequential reads, then
you'll get better performance with the SSD, regardless of your encryption.
For random writes, the SSD will always be better than the HDD, but how many
times faster is determined by support for TRIM.  If you do sequential
writes, then you'll get comparable performance SSD vs HDD when you have no
encryption, or you can support TRIM and background garbage collection.  If
you do sequential writes and cannot support TRIM, then the HDD will probably
outperform the SSD significantly.

Re: Encryption implementation like ZFS?

[Niels de Carpentier]

>> ... and depending on which SSD you use, it shouldn't matter. Really.
>>
>> Last time I tried with sandforce SSD + btrfs + -o discard, forcing
>> trim actually made things slower. Sandforce (and probably other modern
>> SSD) controllers can work just fine even without explicit trim fs
>> support.
>
> What command did you use to test this?
>
> I have an OCZ Agility 3 SSD, which have the latest Sandforce
> controller, so I would really like to try reproduce your test setup.

Ok, the sandforce controller makes things interesting.

First of all, sandforce controllers have a very high failure rate, so make
sure you have backups!!
Sandforce controllers also use compression and deduplication to increase
performance. Encryption will make your data incompressible and random, so
this can have a big impact on performance, depending on the
characteristics of your data.
Sandforce controllers also have life time throttling, which will throttle
writes heavily if it thinks you will wear out the  flash within the
warranty period. If you have a very heavy write workload this can be an
issue.

If you don't have a working trim it is a good idea to leave part of your
drive unused. (Make sure you either do this after a full write erase of
the drive, or do a manual trim of that area, otherwise it won't work).
This will make sure the drive has enough spare sectors to do garbage
collection and can greatly improve performance if your drive is full.

Niels

Re: Encryption implementation like ZFS?

[Fajar A. Nugraha]

On Sun, Jan 1, 2012 at 12:12 AM, Niels de Carpentier
<niels@decarpentier.com> wrote:
>>> ... and depending on which SSD you use, it shouldn't matter. Really=
=2E
>>>
>>> Last time I tried with sandforce SSD + btrfs + -o discard, forcing
>>> trim actually made things slower. Sandforce (and probably other mod=
ern
>>> SSD) controllers can work just fine even without explicit trim fs
>>> support.
>>
>> What command did you use to test this?

Normal usage, and some random i/o test tool like fio.

>>
>> I have an OCZ Agility 3 SSD, which have the latest Sandforce
>> controller, so I would really like to try reproduce your test setup.

Yours should be newer. Mine is somewhat-old corsair force 60 GB with
btrfs on top. When I activated -o discard, it actually become slower.
Also, when I used fstrim, the IOPS were capped at 100, so probably the
slowdown is because of that (i.e. IOPS-limit of TRIM somewhere,
possibly the controller)

>
> Ok, the sandforce controller makes things interesting.
>
> First of all, sandforce controllers have a very high failure rate, so=
 make
> sure you have backups!!

Yes, but even knowing that I can't imagine going back to HDD for this
particular system. It'd be too slow to bear :P

> Sandforce controllers also use compression and deduplication to incre=
ase
> performance. Encryption will make your data incompressible and random=
, so
> this can have a big impact on performance, depending on the
> characteristics of your data.

In my case I use compress=3Dlzo, so it shouldn't be compressible by the
controllers.

> Sandforce controllers also have life time throttling, which will thro=
ttle
> writes heavily if it thinks you will wear out the =A0flash within the
> warranty period. If you have a very heavy write workload this can be =
an
> issue.

That's new. Is there a link/reference for that?

>
> If you don't have a working trim it is a good idea to leave part of y=
our
> drive unused. (Make sure you either do this after a full write erase =
of
> the drive, or do a manual trim of that area, otherwise it won't work)=
=2E
> This will make sure the drive has enough spare sectors to do garbage
> collection and can greatly improve performance if your drive is full.

True. But on my last test I can't get fstrim to trim everything. It
could only trim about 2GB out of 12GB free space.

--=20
=46ajar
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Encryption implementation like ZFS?

[Niels de Carpentier]

> On Sun, Jan 1, 2012 at 12:12 AM, Niels de Carpentier
> <niels@decarpentier.com> wrote:
>>>> ... and depending on which SSD you use, it shouldn't matter. Reall=
y.
>>>>
>>>> Last time I tried with sandforce SSD + btrfs + -o discard, forcing
>>>> trim actually made things slower. Sandforce (and probably other mo=
dern
>>>> SSD) controllers can work just fine even without explicit trim fs
>>>> support.
>>>
>>> What command did you use to test this?
>
> Normal usage, and some random i/o test tool like fio.

Some interesting information here"

http://people.redhat.com/jmoyer/discard/ext4_batched_discard/ext4_disca=
rd.html

and here:

http://purdea.ro/projects/trim_perf/

Depending on the vendor, discard will generally be slower, but can have=
 a
huge performance impact. TRIM is not queueable, so it will force a queu=
e
flush and will block all following commands. The OCZ Agility 3 seems
especially slow, were a discard command will take at least 10ms!

>>> I have an OCZ Agility 3 SSD, which have the latest Sandforce
>>> controller, so I would really like to try reproduce your test setup=
=2E
>
> Yours should be newer. Mine is somewhat-old corsair force 60 GB with
> btrfs on top. When I activated -o discard, it actually become slower.
> Also, when I used fstrim, the IOPS were capped at 100, so probably th=
e
> slowdown is because of that (i.e. IOPS-limit of TRIM somewhere,
> possibly the controller)

See above, do NOT turn on filesytem discard support on a OCZ agility 3,
the performance impact is huge! You're better of scheduling a daily
swiper.sh or fstrim run, with the advantage that it will probably also
work if intermediate layers don't support discard.

>>
>> Ok, the sandforce controller makes things interesting.
>>
>> First of all, sandforce controllers have a very high failure rate, s=
o
>> make
>> sure you have backups!!
>
> Yes, but even knowing that I can't imagine going back to HDD for this
> particular system. It'd be too slow to bear :P

Ik know :) However, considering OCZ sandforce controllers have a failur=
e
rate of 5-10%, compared to intel with 0.5%, you might want to consider
switching to a more reliable one and selling the OCZ. I any case make v=
ery
sure you make frequent backups and that they are working!
The sandforce controllers look nice on paper, but the unreliabilty,
disappointing speeds for compressed data, lifetime write throtteling an=
d
extremely slow trim performance make then one of the worst choices in
practice. The crucial M4 and Samsung 830 are very good in the same pric=
e
class, the intel 320 is the most reliable and a bit cheaper, but slower
and SATA300. The kingston V100+ is a reasonable budget choice. My advic=
e
would be to avoid ocz/sandforce at all costs, you can easily find lots =
of
horror stories with google or newegg product reviews.

>> Sandforce controllers also use compression and deduplication to incr=
ease
>> performance. Encryption will make your data incompressible and rando=
m,
>> so
>> this can have a big impact on performance, depending on the
>> characteristics of your data.
>
> In my case I use compress=3Dlzo, so it shouldn't be compressible by t=
he
> controllers.

Yes, but dedup might still allow the drive to have more empty space to =
do
wear leveling/garbage collection, especially if you don't use trim.
>
>> Sandforce controllers also have life time throttling, which will
>> throttle
>> writes heavily if it thinks you will wear out the =A0flash within th=
e
>> warranty period. If you have a very heavy write workload this can be=
 an
>> issue.
>
> That's new. Is there a link/reference for that?

http://www.xtremesystems.org/forums/showthread.php?272545-Sandforce-Lif=
e-Time-Throttling
http://www.xtremesystems.org/forums/showthread.php?271063-SSD-Write-End=
urance-25nm-Vs-34nm

The endurance test thread is pretty cool, as it shows that SSD's can
handle much more writes then the manufacturer spec (Except for the
sandforce ones, which limit write speeds until it is impossible to exce=
ed
the manufacturer spec during the waranty period.)

> True. But on my last test I can't get fstrim to trim everything. It
> could only trim about 2GB out of 12GB free space.

Did you try wiper.sh? That should work better as it allocates all free
space and then runs trim on it. It might still miss some areas, but sho=
uld
be much better then 2 out of 12!

Niels

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html