Re: Shred mount option for ext4?

Re: Shred mount option for ext4?

[Nikolai Joukov]

It is a surprizing coincidence that Erik posted this request today.  We
also discussed possible ext4 patches for secure deletion yesterday at
StorageSS workshop (www.storagess.org).

> Why don't you just make a libc wrapper for the unlink(2) system call?
> (A modified libc.so should do as well). That way it will work for all
> of your applications on all filesystems.

This won't intercept unlink and truncate requests from statically linked
binaries that you need to intercept.  Users will not trust a solution that
only works in some cases.

> In practice a single overwrite is enough because of the sheer size of
> the task to reproduce the overwritten data.

Agreed.  There is not even a single commercial company that can
recover overwritten data (e.g., see the Action Front (www.actionfront.com)
survey).  It is obvious that data recovery companies tried it and
all failed.  However, some users will believe that governmental labs can
recover overwritten data no matter what you say.  Therefore, there
is a need for two solutions: 1) overwrite the data once and make
it impossible to recover using software tools and all commercially
available methods 2) perform N overwrites for these who want it.

> > Some people (me included) would most likely accept the time penalty of
> > using this option on selected filesystems (as well as the reduced
> > lifetime of the disks because of the extra writes).

We have posted a simple ext3 patch that does one-time overwriting back in
February (http://lwn.net/Articles/171924/).  Valerie Henson made an
interesting point yesterday at StorageSS that we may also need to
overwrite the journal.  Even though the journal gets overwritten quickly
some users will want it to be overwritten for sure.

We plan to post two more patches for secure deletion in ext4 soon.
(Well, as for most of us, it is a side project and the development was not
pretty rapid but now we really plan to release these patches.)  They
already work but still require some polishing:

1. One of the patches performs N overwrites with configurable patterns
(can comply with NIST and NISPOM standards).  Because of the transaction
compaction we had to separately add overwriting as separate transactions.
Fortunately, the whole procedure is still atomic due to the orphan list.
The problem that we have right now is per-file syncing of dirty data
buffers between overwrites.  We sync the whole device at the moment.

2. Another compromise solution is to combine secure deletion with the
trash-bin support (which is requested by even more people :-).  Moving
files on unlink to a special directory is trivial.  Later (e.g., when the
system is idle) a user-mode daemon can just scan this directory,
overwrite, and unlink the files.  For big files it can actually improve
performance.  Also, as Ted T'so suggested for the possible trash bin
patch long ago it may be necessary to add a mechanism for the file system
to initiate the user-mode deletion once the space becomes scarce.  The
benefits of this approach are obvious: 1) small kernel bloat, 2) two
solutions (trash bin and secure deletion) in one, and 3) user-mode part
can be complicated and do overwriting with many patterns, many times, and
at configurable times.  Our current patch makes moved files visible and
accessible by root only.  For a user-friendly trash-bin implementation we
will make files in trash-bin visible for their users.  It will require
readdir and/or lookup changes.

For these who are interested, the preliminary patches are downloadable
from http://www.filesystems.org/project-sdfs.html
We will really appreciate any comments, help, and feedback.

Thank you,
Nikolai Joukov and Harry Papaxenopoulos.
Filesystems and Storage Laboratory,
Stony Brook University

Re: Shred mount option for ext4?

[Andreas Dilger]

On Oct 31, 2006  15:14 -0500, Nikolai Joukov wrote:
> 1. One of the patches performs N overwrites with configurable patterns
> (can comply with NIST and NISPOM standards).  Because of the transaction
> compaction we had to separately add overwriting as separate transactions.
> Fortunately, the whole procedure is still atomic due to the orphan list.
> The problem that we have right now is per-file syncing of dirty data
> buffers between overwrites.  We sync the whole device at the moment.

Did anyone discuss doing this with crypto instead of actually overwriting
the whole file?  It would be pretty easy to store a per-file crypto key
in each inode as an EA, then to "delete" the file all that would be
needed would be to erase the key in a secure matter (which is a great
deal easier because inodes don't move around on disk).

The drawback is there is a runtime overhead to encrypt/decrypt the file
data, but honestly, if people care about secure deletion don't they also
care about security of the undeleted data also?  By having an (unknown
to the user) per-file crypto key then if the file is deleted the user
can also plausibly deny the ability to recover the file data even if
they are forced to surrender their key.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: Shred mount option for ext4?

[Ric Wheeler]

Andreas Dilger wrote:

>On Oct 31, 2006  15:14 -0500, Nikolai Joukov wrote:
>  
>
>>1. One of the patches performs N overwrites with configurable patterns
>>(can comply with NIST and NISPOM standards).  Because of the transaction
>>compaction we had to separately add overwriting as separate transactions.
>>Fortunately, the whole procedure is still atomic due to the orphan list.
>>The problem that we have right now is per-file syncing of dirty data
>>buffers between overwrites.  We sync the whole device at the moment.
>>    
>>
>
>Did anyone discuss doing this with crypto instead of actually overwriting
>the whole file?  It would be pretty easy to store a per-file crypto key
>in each inode as an EA, then to "delete" the file all that would be
>needed would be to erase the key in a secure matter (which is a great
>deal easier because inodes don't move around on disk).
>  
>
This is an interesting idea with some annoying implementation details. 
For example, we would still need to "shred" that data block used to 
store the EA in order to prevent key recovery.

Also interesting to note that various people are putting encryption into 
various offload parts which could be useful in this context.

>The drawback is there is a runtime overhead to encrypt/decrypt the file
>data, but honestly, if people care about secure deletion don't they also
>care about security of the undeleted data also?  By having an (unknown
>to the user) per-file crypto key then if the file is deleted the user
>can also plausibly deny the ability to recover the file data even if
>they are forced to surrender their key.
>
>Cheers, Andreas
>--
>
>  
>
I think that having the data encrypted on disk is a generically useful 
feature, but in this case it might not count for much since the key is 
stored right next to the data in that EA...

ric

Re: Shred mount option for ext4?

[Nikolai Joukov]

> >>1. One of the patches performs N overwrites with configurable patterns
> >>(can comply with NIST and NISPOM standards).  Because of the transaction
> >>compaction we had to separately add overwriting as separate transactions.
> >>Fortunately, the whole procedure is still atomic due to the orphan list.
> >>The problem that we have right now is per-file syncing of dirty data
> >>buffers between overwrites.  We sync the whole device at the moment.
> >
> >Did anyone discuss doing this with crypto instead of actually overwriting
> >the whole file?  It would be pretty easy to store a per-file crypto key
> >in each inode as an EA, then to "delete" the file all that would be
> >needed would be to erase the key in a secure matter (which is a great
> >deal easier because inodes don't move around on disk).

Encryption is another possible secure deletion solution.  Usually it is
used by systems that already encrypt the data anyways.  In that case the
key management and run-time overhead costs are already paid.

> >The drawback is there is a runtime overhead to encrypt/decrypt the file

The difference is that in case of encryption there are overheads for read
and write operations whereas in case of overwriting there are overheads
only for infrequent unlink/truncate operations.

> I think that having the data encrypted on disk is a generically useful
> feature, but in this case it might not count for much since the key is
> stored right next to the data in that EA...

Agreed.  Key management is a big issue in any encryption system.  In this
particular solution the key management is simple but there is also no
real protection of the live data.

Nikolai.

Re: Shred mount option for ext4?

[Erez Zadok]

Also of note is that some government standards for data destruction (e.g.,
NIST and NISPOM) specifically require that files be overwritten with a
certain pattern, and even multiple times for different security levels.  So
those sites who have to comply with such standards won't be able to use an
encryption-only solution to discard data.

The other problem with crypto on persistent media is that you can't easily
predict how secure data will be in the really long run, say years or
decades.  Suppose an attacker manages to get their hands on a crypto disk
whose keys were destroyed.  With today's technology, the attacker might not
be able to break the crypto.  But wait 10 years, and who knows what kinds of
computing powers we will have then.  (Even ten-year old data, if recovered,
could still be highly valuable -- eg., top-secret military files).

Crypto is great.  But overwriting data even just a couple of times w/ random
bits, elevates the level of attack required to recover such data, well past
crypto.

Erez.

RE: Shred mount option for ext4?

[Wolber, Richard C]

> On Wednesday, November 01, 2006 8:17 AM Andreas Dilger Wrote:
> 
> Did anyone discuss doing this with crypto instead of actually 
> overwriting the whole file?  It would be pretty easy to store 
> a per-file crypto key in each inode as an EA, then to 
> "delete" the file all that would be needed would be to erase 
> the key in a secure matter (which is a great deal easier 
> because inodes don't move around on disk).

If it's cheap to delete the keys, it's also cheap to harvest 
the keys. A per file crypto-key lowers the barrier to entry.

This is Schneier 101.


..Chuck..

Re: Shred mount option for ext4?

[Rupesh Thakare]

Wolber, Richard C wrote:
>> On Wednesday, November 01, 2006 8:17 AM Andreas Dilger Wrote:
>>
>> Did anyone discuss doing this with crypto instead of actually 
>> overwriting the whole file?  It would be pretty easy to store 
>> a per-file crypto key in each inode as an EA, then to 
>> "delete" the file all that would be needed would be to erase 
>> the key in a secure matter (which is a great deal easier 
>> because inodes don't move around on disk).
>>     
>
> If it's cheap to delete the keys, it's also cheap to harvest 
> the keys. A per file crypto-key lowers the barrier to entry.
>   
That's true. But can't we combine the advantages of single-secure-key 
and per-file krypto key ?
Can't we have a half single-secure-key combined with half 
per-file-krypto ? Key management overhead is not
worse than that for single-secure-key. This gives offers same security 
with ease for shredding.

Cheers,
Rupesh
> This is Schneier 101.
>
>
> ..Chuck..
> -
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

Shred mount option for ext4?

[Samuel Tardieu]

Sorry if this has been discussed already, I couldn't find anything in
the list archive about it. What about adding the possibility of
shredding or erasing free blocks on an ext4 filesystem?

I value my privacy and the privacy of the people I host, and I often
use shred(1) when erasing files from my server. The goal is to avoid
that either a hacker or a post-mortem analysis gets ancien data from
my disk. There are three problems with this approach:

  - I may forget to use shred sometimes

  - some files are automatically created and then removed (mails in
    spool)

  - data may be replicated in the journal and thus still present on
    disk

I could use an encrypted filesystem everywhere, but in many countries,
one is required to reveal her encryption key to authorities if they
have a court order (UK for example).

I think it would be quite easy to add a mount time option to ext4
filesystems asking that freed blocks are cleared or erased with random
data? We could have for example:

  - free=clear|zero|shred (default "clear", do nothing, "zero" means
    writing zeroes over the block, useful against attackers trying to
    recover data from a file system without physical access to it, and
    "shred" useful against post-mortem analysis of the physical
    surface)

  - shred-passes=N (number of passes when using the "free=shred"
    option, a negative number meaning writing values from 0 to -N onto
    the block)

Some people (me included) would most likely accept the time penalty of
using this option on selected filesystems (as well as the reduced
lifetime of the disks because of the extra writes).

I would contribute a proof-of-concept code, but I'm going to leave for
a one-month vacation and will have a very bad connection until
December. However, if noone jumps on that, I will likely code that
when I go back unless someone beats me on it. In the meantime, I'd
like to get people thoughts about it.

  Sam
-- 
Samuel Tardieu -- sam@rfc1149.net -- http://www.rfc1149.net/

Re: Shred mount option for ext4?

[Erik Mouw]

On Tue, Oct 31, 2006 at 11:36:11AM +0100, Samuel Tardieu wrote:
> I value my privacy and the privacy of the people I host, and I often
> use shred(1) when erasing files from my server. The goal is to avoid
> that either a hacker or a post-mortem analysis gets ancien data from
> my disk. There are three problems with this approach:
> 
>   - I may forget to use shred sometimes
> 
>   - some files are automatically created and then removed (mails in
>     spool)
> 
>   - data may be replicated in the journal and thus still present on
>     disk
> 
> I could use an encrypted filesystem everywhere, but in many countries,
> one is required to reveal her encryption key to authorities if they
> have a court order (UK for example).

Interesting. I'd say that you don't have to cooperate to get yourself
convicted (i.e.: the right to remain silent). Over here in .nl you do
have to cooperate to decrypt data from a suspect other than yourself,
but you don't have to for your own data.

> I think it would be quite easy to add a mount time option to ext4
> filesystems asking that freed blocks are cleared or erased with random
> data? We could have for example:
> 
>   - free=clear|zero|shred (default "clear", do nothing, "zero" means
>     writing zeroes over the block, useful against attackers trying to
>     recover data from a file system without physical access to it, and
>     "shred" useful against post-mortem analysis of the physical
>     surface)
> 
>   - shred-passes=N (number of passes when using the "free=shred"
>     option, a negative number meaning writing values from 0 to -N onto
>     the block)

FWIW, the idea that you need to rewrite 35 times doesn't longer hold.
Modern drives use PRML encoding techniques, so a few random writes are
enough if you're really paranoid. See the "Epilogue" section in
Gutmann's paper at
http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html .

In practice a single overwrite is enough because of the sheer size of
the task to reproduce the overwritten data. Compare it with a painting
that was "overwritten" with white paint. Sure when you use a microscope
you might be able to figure out some of the original color of the paint
below the white, but it will take years to reproduce it. So far we
haven't found a customer willing to wait a few years for his data...

> Some people (me included) would most likely accept the time penalty of
> using this option on selected filesystems (as well as the reduced
> lifetime of the disks because of the extra writes).

Why don't you just make a libc wrapper for the unlink(2) system call?
(A modified libc.so should do as well). That way it will work for all
of your applications on all filesystems.


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

Re: Shred mount option for ext4?

[Samuel Tardieu]

On 31/10, Erik Mouw wrote:

| Why don't you just make a libc wrapper for the unlink(2) system call?
| (A modified libc.so should do as well). That way it will work for all
| of your applications on all filesystems.

Because this won't take care of the journal, which may contain an extra
copy of the previously written data. Also in transactional filesystems,
overwriting data doesn't guarantee that the data is written in the same
place: it will typically be written at another place and then the
metadata will be updated afterwards. You have no control about that at
user level, since you can't hook into the block freeing routine.

Thanks for the reference btw :)