Storage system

Storage system

[Piergiorgio Sartor]

Hi all,

this question is only partially related to Linux MD,
but since the experts are here, I think it would not
be a big problem to ask here.

I'm considering a storage system.
This is based on HDD "rust".
It should have RAID-6, for protection agaist disk
failure(s).
It should have LUKS (or similar), in order to simplify
HDD disposal (disk that are still somehow readable will
not need to be wiped out before dumping them).
It should have LVM, as flexible partition system.

Now, given that the HDD are always the lower layer, the
others can be combined in any order.
Six possible combinations, specifically, naming:

H: HDD
R: RAID
E: encryption
L: LVM

We can have:

HREL
HRLE
HLRE
HLER
HERL
HELR

Those are all possible (and valid) configurations.
Some are "better" and some are less "good", depending
on "good" or "bad" definition.
For example, in term of pure performances, I tested
HREL vs. HERL, the first is quite faster than the
second one.
On the other hand, HERL encrypts the RAID header, so
delivers less information to whoever wants to check
the disk content.

HRLE gives the possibility of having volumes encrypted
separately, allowing a finer security control.

HLxy, I think, do not really offer any advantage, but
I'm not sure.

So, finally, the question would be:

Knowing that it depends on requirements, do you think
one of this configuration is better than the other?
Why?

Feel free to elaborate, if you like.

Thanks a lot in advance,

-- 

piergiorgio

Re: Storage system

[Joe Landman]

On 02/06/2014 01:09 PM, Piergiorgio Sartor wrote:
> Hi all,
>
> this question is only partially related to Linux MD,
> but since the experts are here, I think it would not
> be a big problem to ask here.
>
> I'm considering a storage system.
> This is based on HDD "rust".
> It should have RAID-6, for protection agaist disk
> failure(s).
> It should have LUKS (or similar), in order to simplify
> HDD disposal (disk that are still somehow readable will
> not need to be wiped out before dumping them).

Can you use the full disk encryption mechanisms?  This could 
(potentially) save a layer, though the down side is that you don't have 
control over it.  We've not used it, most of our customers destroying 
disks tend to physically destroy them, and not trust the encryption layer.

> It should have LVM, as flexible partition system.
>
> Now, given that the HDD are always the lower layer, the
> others can be combined in any order.
> Six possible combinations, specifically, naming:
>
> H: HDD
> R: RAID
> E: encryption
> L: LVM
>
> We can have:
>
> HREL
> HRLE
> HLRE
> HLER
> HERL
> HELR

If this is a heavily used system, you'll likely want to start with HR 
for performance reasons.  If you can eliminate the E with FDE as noted, 
and if it works well, great.  If not, I'd suggest putting that next. 
LVM isn't a fast system, so its better not to put it on a fast system. 
LUKS can be fast with good CPUs and reasonable controllers/disks.

Re: Storage system

[David Brown]

On 06/02/14 19:09, Piergiorgio Sartor wrote:
> Hi all,
> 
> this question is only partially related to Linux MD,
> but since the experts are here, I think it would not
> be a big problem to ask here.
> 
> I'm considering a storage system.
> This is based on HDD "rust".
> It should have RAID-6, for protection agaist disk
> failure(s).
> It should have LUKS (or similar), in order to simplify
> HDD disposal (disk that are still somehow readable will
> not need to be wiped out before dumping them).
> It should have LVM, as flexible partition system.
> 


It strikes me as a bad idea to use encryption of any sort "to save time
when dumping old disks".  Physically destroying hard disks is not /that/
hard.  Unless you are keeping plans for a nuclear missile, then a few
whacks with a hammer will be good enough.  Breaking the electronics
means it costs many thousands of dollars to get the data off the disk
again - you don't even need to open the drive and get out the platters
(opening the drive is time-consuming - destroying the platters after
opening is easy).  And with raid, little of the data on the disk is
intelligible unless you have the full stripe (minus parity) - just ask
anyone who has tried to recover from one too many disk failures.

And of course, just dd'ing /dev/zero to the first few MB of the disk
will make it unreadable for most hackers - even if they have all the
disks in a set, and know how they were configured.  And you could donate
the old disks to windows users - then they are guaranteed unreadable!

Disk encryption slows everything down, and adds lots of complications to
the system.  It is less of an issue with drives with built-in
encryption, but still a complete waste of time and money if all you want
is "safe" disposal of old disks.

The /only/ thing disk encryption is useful for is if you fear the disks
will be physically stolen by someone who is after your data (or customs
guards in dodgy countries, which amounts to the same thing).  So if you
fear that your company will be the target of top-range thieves who will
steal your disks for the data, then encryption is a good idea.  Of
course, better locks and alarm systems would be a better investment.


Once you have eliminated the "E", then I believe HRL is the common
arrangement, although sometimes you also do physical partitioning of the
disks first, so that you can have different bits with different raid
types.  A multi-way raid1 partition first for /boot can make booting
easier, a set of raid1 pairs works well for swap (for emergency use
only), and then the rest of each disk makes up your raid6 array.

Re: Storage system

[Matt Garman]

FWIW, I use a program called "shred" when I'm done with a disk.  It
makes N (default = 3) passes of writing random data to the disk, and
an optional final pass of zeroes.  It's time-consuming to complete,
but takes only 30 seconds to get going.  Even more convenient if you
have one of those USB hard drive docks, so you can take the drive out
of your system.

Based on what I've read, that should be sufficient to keep anyone
without a Dept of Defense budget from recovering you data.  The DOD
probably already has your data anyway, so that's a non-issue.  :)



On Fri, Feb 7, 2014 at 2:07 AM, David Brown <david.brown@hesbynett.no> wrote:
> On 06/02/14 19:09, Piergiorgio Sartor wrote:
>> Hi all,
>>
>> this question is only partially related to Linux MD,
>> but since the experts are here, I think it would not
>> be a big problem to ask here.
>>
>> I'm considering a storage system.
>> This is based on HDD "rust".
>> It should have RAID-6, for protection agaist disk
>> failure(s).
>> It should have LUKS (or similar), in order to simplify
>> HDD disposal (disk that are still somehow readable will
>> not need to be wiped out before dumping them).
>> It should have LVM, as flexible partition system.
>>
>
>
> It strikes me as a bad idea to use encryption of any sort "to save time
> when dumping old disks".  Physically destroying hard disks is not /that/
> hard.  Unless you are keeping plans for a nuclear missile, then a few
> whacks with a hammer will be good enough.  Breaking the electronics
> means it costs many thousands of dollars to get the data off the disk
> again - you don't even need to open the drive and get out the platters
> (opening the drive is time-consuming - destroying the platters after
> opening is easy).  And with raid, little of the data on the disk is
> intelligible unless you have the full stripe (minus parity) - just ask
> anyone who has tried to recover from one too many disk failures.
>
> And of course, just dd'ing /dev/zero to the first few MB of the disk
> will make it unreadable for most hackers - even if they have all the
> disks in a set, and know how they were configured.  And you could donate
> the old disks to windows users - then they are guaranteed unreadable!
>
> Disk encryption slows everything down, and adds lots of complications to
> the system.  It is less of an issue with drives with built-in
> encryption, but still a complete waste of time and money if all you want
> is "safe" disposal of old disks.
>
> The /only/ thing disk encryption is useful for is if you fear the disks
> will be physically stolen by someone who is after your data (or customs
> guards in dodgy countries, which amounts to the same thing).  So if you
> fear that your company will be the target of top-range thieves who will
> steal your disks for the data, then encryption is a good idea.  Of
> course, better locks and alarm systems would be a better investment.
>
>
> Once you have eliminated the "E", then I believe HRL is the common
> arrangement, although sometimes you also do physical partitioning of the
> disks first, so that you can have different bits with different raid
> types.  A multi-way raid1 partition first for /boot can make booting
> easier, a set of raid1 pairs works well for swap (for emergency use
> only), and then the rest of each disk makes up your raid6 array.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Storage system

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 1481 bytes --]

On Fri, 7 Feb 2014 08:29:19 -0600
Matt Garman <matthew.garman@gmail.com> wrote:

> FWIW, I use a program called "shred" when I'm done with a disk.

Disks are not always alive when they need to be thrown out, I'd say it's very
much to the contrary. And while the drive electronics may be bust so it
doesn't even detect in the computer, all your data could still sit perfectly
intact on the plates.

Of course a physical destruction of the disk is an option in this case, but
what if instead of throwing away such a disk, you need to RMA it to the
vendor (or send to a data recovery company)? It's the absolutely worst case:
an untold number of people will have unlimited access to your data after they
repair the drive, for as long as they want (they can make a full copy).

Having said all of the above, I do think it is an overkill to use encryption
above/below RAID5/6 to protect against such cases. I feel that "they" get only
a crude haircomb-like view of the data (every 6th 64KB or whatever) is enough
to effectively make the data unusable.

Of course there's still 1 in 6 chance, for a 6-device RAID, that your
passwords, or your private ssh key, or whatever, will fall entirely into that
64KB portion that was on the drive in question, and someone will be
determined-enough that in the absense of a working filesystem look through
all of the drive and find it. But I think that's a risk we can learn to live
with. :)

-- 
With respect,
Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Storage system

[Roberto Spadim]

hum... IMHO...
security isn't cheap if you want performace, right?

with a disk with important data, I don't do rma
buy another and replace, it's cheap, less time lost, and more
performace without encryption, ok if you lost data, why you don't do
backup and mirror, maybe replication (databases)? replace the bad disk
and system works, restore backup and start jobs again

ok, but if you want crypt at disk? add a layer at filesystem is the
best option check:
1)HERL
if you have a crash at crypt, you lost raid and lvm, how to restore a
file if you can't start raid and lvm? you will restore the full disk
(terabytes) or you prefer restore some files (kb, mb, gb?)

2)HRLE
if you have a crash at crypt, you lost some files, but your lvm still
running and your raid system too

ok when you crash a disk you lost many things, but some things without
encryption are easier to read, data without crypt could be "HELLO
GUY", data with crypt coudl be "ASDFJWE", now, you remember that you
have a filename with "HELLO GUY", try to find it with a crash disk
with crypt, and without, without crypt is easier, right?

i think the best option is crypt at filesystem, and don't crypt
metadata (file name for example), just the data (file contents),
that's my opnion

Re: Storage system

[David Brown]

On 07/02/14 15:29, Matt Garman wrote:
> FWIW, I use a program called "shred" when I'm done with a disk.  It
> makes N (default = 3) passes of writing random data to the disk, and
> an optional final pass of zeroes.  It's time-consuming to complete,
> but takes only 30 seconds to get going.  Even more convenient if you
> have one of those USB hard drive docks, so you can take the drive out
> of your system.
> 
> Based on what I've read, that should be sufficient to keep anyone
> without a Dept of Defense budget from recovering you data.  The DOD
> probably already has your data anyway, so that's a non-issue.  :)
> 

Even with the full resources of NSA, there is no way to read useful data
from a disk that has had a single pass of writing zeros.  Multi-pass
disk erasure is a case of some companies getting very rich from a myth
based on a single academic paper with theories about how /some/ data
/might/ be recoverable from overwritten disks.  It was never more than
an idea at the time, and it applies even less to modern disks.

In an experiment, researchers wrote some bits to a sample of hard drive
material, overwrote them once, then tried to read the old data.  I can't
find the reference (I really wish I could), so my figures may be a bit
off - but they are in the right ballpark.  I believe it was about 32
bits they wrote.  They managed to recover 7 bits that they were
confident were correct - after spending months with equipment such as
electron microscopes.

The most convincing argument, however, is the economic one - if it were
possible to recover erased data (that has been overwritten once) with
any reliability, then disk recovery firms would be offering such a
service.  These folks can get your photos back after disks have been
"destroyed" in fires - but they cannot get data back if it has been erased.


The only thing you can't erase are re-mapped sectors - but hopefully
those will be rare!

With SSD's it's a bit different - re-mapping and garbage collection
means that there is the possibility of some data being left on partially
filled blocks.

> 
> 
> On Fri, Feb 7, 2014 at 2:07 AM, David Brown <david.brown@hesbynett.no> wrote:
>> On 06/02/14 19:09, Piergiorgio Sartor wrote:
>>> Hi all,
>>>
>>> this question is only partially related to Linux MD,
>>> but since the experts are here, I think it would not
>>> be a big problem to ask here.
>>>
>>> I'm considering a storage system.
>>> This is based on HDD "rust".
>>> It should have RAID-6, for protection agaist disk
>>> failure(s).
>>> It should have LUKS (or similar), in order to simplify
>>> HDD disposal (disk that are still somehow readable will
>>> not need to be wiped out before dumping them).
>>> It should have LVM, as flexible partition system.
>>>
>>
>>
>> It strikes me as a bad idea to use encryption of any sort "to save time
>> when dumping old disks".  Physically destroying hard disks is not /that/
>> hard.  Unless you are keeping plans for a nuclear missile, then a few
>> whacks with a hammer will be good enough.  Breaking the electronics
>> means it costs many thousands of dollars to get the data off the disk
>> again - you don't even need to open the drive and get out the platters
>> (opening the drive is time-consuming - destroying the platters after
>> opening is easy).  And with raid, little of the data on the disk is
>> intelligible unless you have the full stripe (minus parity) - just ask
>> anyone who has tried to recover from one too many disk failures.
>>
>> And of course, just dd'ing /dev/zero to the first few MB of the disk
>> will make it unreadable for most hackers - even if they have all the
>> disks in a set, and know how they were configured.  And you could donate
>> the old disks to windows users - then they are guaranteed unreadable!
>>
>> Disk encryption slows everything down, and adds lots of complications to
>> the system.  It is less of an issue with drives with built-in
>> encryption, but still a complete waste of time and money if all you want
>> is "safe" disposal of old disks.
>>
>> The /only/ thing disk encryption is useful for is if you fear the disks
>> will be physically stolen by someone who is after your data (or customs
>> guards in dodgy countries, which amounts to the same thing).  So if you
>> fear that your company will be the target of top-range thieves who will
>> steal your disks for the data, then encryption is a good idea.  Of
>> course, better locks and alarm systems would be a better investment.
>>
>>
>> Once you have eliminated the "E", then I believe HRL is the common
>> arrangement, although sometimes you also do physical partitioning of the
>> disks first, so that you can have different bits with different raid
>> types.  A multi-way raid1 partition first for /boot can make booting
>> easier, a set of raid1 pairs works well for swap (for emergency use
>> only), and then the rest of each disk makes up your raid6 array.
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Storage system

[Can Jeuleers]

On 02/07/2014 05:11 PM, David Brown wrote:
> The only thing you can't erase are re-mapped sectors - but hopefully
> those will be rare!

I wonder if using the Secure Erase feature destroys all sectors
(including remapped-ones) or not.

Also, what level of trust do people on this list have that the disk's
Secure Erase feature does what it says on the tin?

Re: Storage system

[David Brown]

On 07/02/14 17:25, Can Jeuleers wrote:
> On 02/07/2014 05:11 PM, David Brown wrote:
>> The only thing you can't erase are re-mapped sectors - but hopefully
>> those will be rare!
> 
> I wonder if using the Secure Erase feature destroys all sectors
> (including remapped-ones) or not.
> 

I think secure erase can be done in different ways.  On disks that have
full-disk encryption, it simply erases (or changes) the password.  For
other drives, it has to write through all sectors on the disk - I would
expect re-mapped sectors are included, but I don't know the details.


> Also, what level of trust do people on this list have that the disk's
> Secure Erase feature does what it says on the tin?

Re: Storage system

[Chris Murphy]

On Feb 7, 2014, at 9:36 AM, David Brown <david.brown@hesbynett.no> wrote:

> On 07/02/14 17:25, Can Jeuleers wrote:
>> On 02/07/2014 05:11 PM, David Brown wrote:
>>> The only thing you can't erase are re-mapped sectors - but hopefully
>>> those will be rare!
>> 
>> I wonder if using the Secure Erase feature destroys all sectors
>> (including remapped-ones) or not.
>> 
> 
> I think secure erase can be done in different ways.  On disks that have
> full-disk encryption, it simply erases (or changes) the password.  For
> other drives, it has to write through all sectors on the disk - I would
> expect re-mapped sectors are included, but I don't know the details.

From the September 2012 draft, rev 1 of NIST 800-80, they distinguish between the following ATA commands:

	• ATA sanitize command
	• ATA Secure Erase command
	• Cryptographic Erase

I don't have a clear understanding of what "sanitize" command is, although it's relatively new. None of my drives have it. hdparm reports my drives support SECURITY ERASE UNIT and ENHANCED SECURITY ERASE UNIT. For HDDs, these take the same time. The first one writes zeros. The 2nd one writes a manufacturer defined pattern (i.e. it's almost certainly not random). The Samsung 830 SSD also has these two commands, but the time difference between them was substantial: I *think* 3 minutes for one and 18 minutes for the other.

Cryptographic erase doesn't overwrite anything. It just makes sure the MEKs are changed, thereby making the drive unreadable, since SEDs are always encrypting using a MEK, even if there isn't a KEK set.


> 
> 
>> Also, what level of trust do people on this list have that the disk's
>> Secure Erase feature does what it says on the tin?\

Again NIST 800-80 says: "Verification must be performed for each technique within Clear and Purge, except degaussing." Strictly speaking the trust is zero without verification.

Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Storage system

[Robert L Mathews]

On 2/7/14, 8:11 AM, David Brown wrote:

> In an experiment, researchers wrote some bits to a sample of hard drive
> material, overwrote them once, then tried to read the old data.  I can't
> find the reference (I really wish I could)

Overview at:

 http://digital-forensics.sans.org/blog/2009/01/15/overwriting-hard-drive-data/

Detailed article at:

 http://privazer.com/overwriting_hard_drive_data.The_great_controversy.pdf

-- 
Robert L Mathews, Tiger Technologies, http://www.tigertech.net/

Re: Storage system

[Chris Murphy]

On Feb 7, 2014, at 7:29 AM, Matt Garman <matthew.garman@gmail.com> wrote:

> FWIW, I use a program called "shred" when I'm done with a disk.  It
> makes N (default = 3) passes of writing random data to the disk, and
> an optional final pass of zeroes.  It's time-consuming to complete,
> but takes only 30 seconds to get going. 

Unnecessary. If the drive is alive, use ATA Security Erase with hdparm. The drive then zeros itself. NIST 800-88 lists three levels of sanitization, in order of thoroughness: clear, purge, destroy. Using software utilities to write zeros or a pattern, even multiple times, only qualifies as clear. One ATA Security Erase qualifies for purge. And destroy means to literally shred, disintegrate, pulverize or incinerate.

The main reason ATA Security Erase achieves higher level of sanitization is that it will zero dereferenced sectors - those that don't have an LBA assigned to them. For example, bad sectors which contain latent data but have been removed from use. Software can't zero those because they have no LBA to instruct the drive to overwrite.

ATA Security Erase is also faster, and there's no CPU load.

Chris Murphy