Re: secure erasure of files?

Re: secure erasure of files?

[Roy Sigurd Karlsbakk]

> IMHO overwriting with /dev/zero or /dev/random is sufficient.
> Recovering data after that falls into urban legend category :-)

I know of personal experience that the company ibas (http://www.ibas.com)
have, in lab, recovered data overwritten >30 times. To recover data
overwritten from /dev/zero is done in minutes.

roy

--
Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA

Computers are like air conditioners.
They stop working when you open Windows.

Re: secure erasure of files?

[Jan-Frode Myklebust]

On Tue, Feb 12, 2002 at 02:33:47PM +0100, Roy Sigurd Karlsbakk wrote:
> > IMHO overwriting with /dev/zero or /dev/random is sufficient.
> > Recovering data after that falls into urban legend category :-)
> 
> I know of personal experience that the company ibas (http://www.ibas.com)
> have, in lab, recovered data overwritten >30 times. To recover data
> overwritten from /dev/zero is done in minutes.
> 

Interessting, but according to the following newsposting (in
Norwegian) IBAS is clearly stating that they don't know of any
documented methods to read back overwritten data, or know of anyone
who are able to do this.


   -jf

-----------------------------------------------------------------------------
Path: nntp.uib.no!uio.no!nntp.uio.no!not-for-mail
From: "Erik Andersen" <Erik@Andersen.tf>
Newsgroups: no.fag.jus.it
Subject: Re: Loggføring av bevegelser på  Internett
Date: Thu, 22 Mar 2001 10:17:50 +0100
Message-ID: <99cg5n$8ur$1@readme.uio.no>
Reply-To: "Erik Andersen" <Erik@Andersen.tf>
Xref: nntp.uib.no no.fag.jus.it:387

Med tillatelse fra FoU-sjefen gjengir jeg hans svar i sin helhet:



Jeg skal forsøke å svare på dine spørsmål:

Det korte svaret er: Nei det er ikke mulig å lese data som virkelig fysisk
er blitt overskrevet.

Imidlertid er grunnen til dette litt annerledes en det du beskriver.  For å
snakke fornuftig om dette er det først nødvendig med forståelse av hva et
bit på en HD er. En HD opererer ikke med individuelle bit, men med
flux-endringer. Flux retning er enkelt fortalt hvorvidt magnet-feltet på
disken peker mot eller med klokka (CW eller CCW). Så en flux endring er
altså en endring fra f.eks CW til CCW flux retning. Mapping mellom flux
endringer er ikke en-til-en. Det betyr at man IKKE benytter CW=0, CCW=1. I
stedet gir en enkelt.flux-endring opphav til 2.5 til 3 bit. I tillegg
benytter disken sekvens detektering. Dvs. at den ikke prøver å dekode
bit'ene hver for seg, men i stedet ser på en hel sekvens (typisk 4096 bit =
sektor).

Denne sekvensdetekteringen disken gjør ligner mye på hvordan vi leser en
dårlig telefax. Hvis vi forsøker å lese faxen bokstav for bokstav kan vi
f.eks lett forveksle en a med en o. Hvis denne bokstaven er en del av ordet
'bank', og vi tolker bokstav for bokstav ender vi med ordet 'bonk'.  Hvis vi
ser på hele ordet (sekvensen med bokstaver) kan vi se at det mest
sannsynlige ordet er 'bank'.

Det man kan si er at man etter en overskriving kan måle hvor sterke de
gamle dataene er i forhold til de nye. Det betyr at alle 'gamle' signaler
faktisk ikke forsvinner. Våre undersøkelser viser imidlertid at det ikke
finnes noen beskrivelser i litteraturen om hvordan man kan omdanne disse
signalrestene til de opprinnelige dataene.

Det kan synes som at dette krever banebrytende oppdagelser i en rekke
disipliner: Ikke-linjær analyse og modellering, lavt-støyende elektronikk
(cryo-elektronikk), datamaskin teknologi (superraske tallknusere).

Og det var det lange (kompliserte) svaret :)

Det som er sikkert: Ibas kjenner ikke til dokumenterte metoder,
vitenskapelige miljøer eller kommersielle tjenester som utføre eller
demonstrere lesing av overskrevne data.

--
Thor Arne Johansen
Avdelingssjef FoU, Ibas AS



Han har nå lagt til følgende:


Det er imidlertid på sin plass å nevne at dette er et tema
hvor de 'lærde' strides. Dvs. det finnes enkelte som mener at det er mulig
å lese overskrevne data. Men vi har som sagt ikke kunnet finne noe
vitenskaplig dokumentasjon eller beskrivelser av hvordan dette kan
gjøres.

-----------------------------------------------------------------------------


  -jf

Re: secure erasure of files?

[Jens Schmidt]

Hi All,

The information in this message is very interesting, and provided me with
a real insight in the matter of magnetic media.

I was able to read the content, and hopefully understand it correctly,
however, I would not presume to provide an authoritative translation
into English.

I would strongly encourage somebody with fluent Norsk/English skills
to do a translation and post it to the list.

Thank You

Regards Jens Schmidt

Jan-Frode Myklebust wrote:

> On Tue, Feb 12, 2002 at 02:33:47PM +0100, Roy Sigurd Karlsbakk wrote:
> > > IMHO overwriting with /dev/zero or /dev/random is sufficient.
> > > Recovering data after that falls into urban legend category :-)
> >
> > I know of personal experience that the company ibas (http://www.ibas.com)
> > have, in lab, recovered data overwritten >30 times. To recover data
> > overwritten from /dev/zero is done in minutes.
> >
>
> Interessting, but according to the following newsposting (in
> Norwegian) IBAS is clearly stating that they don't know of any
> documented methods to read back overwritten data, or know of anyone
> who are able to do this.
>
>    -jf
>
> -----------------------------------------------------------------------------
> Path: nntp.uib.no!uio.no!nntp.uio.no!not-for-mail
> From: "Erik Andersen" <Erik@Andersen.tf>
> Newsgroups: no.fag.jus.it
> Subject: Re: Loggføring av bevegelser på  Internett
> Date: Thu, 22 Mar 2001 10:17:50 +0100
> Message-ID: <99cg5n$8ur$1@readme.uio.no>
> Reply-To: "Erik Andersen" <Erik@Andersen.tf>
> Xref: nntp.uib.no no.fag.jus.it:387
>
> Med tillatelse fra FoU-sjefen gjengir jeg hans svar i sin helhet:
>
> Jeg skal forsøke å svare på dine spørsmål:
>
> Det korte svaret er: Nei det er ikke mulig å lese data som virkelig fysisk
> er blitt overskrevet.
>
> Imidlertid er grunnen til dette litt annerledes en det du beskriver.  For å
> snakke fornuftig om dette er det først nødvendig med forståelse av hva et
> bit på en HD er. En HD opererer ikke med individuelle bit, men med
> flux-endringer. Flux retning er enkelt fortalt hvorvidt magnet-feltet på
> disken peker mot eller med klokka (CW eller CCW). Så en flux endring er
> altså en endring fra f.eks CW til CCW flux retning. Mapping mellom flux
> endringer er ikke en-til-en. Det betyr at man IKKE benytter CW=0, CCW=1. I
> stedet gir en enkelt.flux-endring opphav til 2.5 til 3 bit. I tillegg
> benytter disken sekvens detektering. Dvs. at den ikke prøver å dekode
> bit'ene hver for seg, men i stedet ser på en hel sekvens (typisk 4096 bit =
> sektor).
>
> Denne sekvensdetekteringen disken gjør ligner mye på hvordan vi leser en
> dårlig telefax. Hvis vi forsøker å lese faxen bokstav for bokstav kan vi
> f.eks lett forveksle en a med en o. Hvis denne bokstaven er en del av ordet
> 'bank', og vi tolker bokstav for bokstav ender vi med ordet 'bonk'.  Hvis vi
> ser på hele ordet (sekvensen med bokstaver) kan vi se at det mest
> sannsynlige ordet er 'bank'.
>
> Det man kan si er at man etter en overskriving kan måle hvor sterke de
> gamle dataene er i forhold til de nye. Det betyr at alle 'gamle' signaler
> faktisk ikke forsvinner. Våre undersøkelser viser imidlertid at det ikke
> finnes noen beskrivelser i litteraturen om hvordan man kan omdanne disse
> signalrestene til de opprinnelige dataene.
>
> Det kan synes som at dette krever banebrytende oppdagelser i en rekke
> disipliner: Ikke-linjær analyse og modellering, lavt-støyende elektronikk
> (cryo-elektronikk), datamaskin teknologi (superraske tallknusere).
>
> Og det var det lange (kompliserte) svaret :)
>
> Det som er sikkert: Ibas kjenner ikke til dokumenterte metoder,
> vitenskapelige miljøer eller kommersielle tjenester som utføre eller
> demonstrere lesing av overskrevne data.
>
> --
> Thor Arne Johansen
> Avdelingssjef FoU, Ibas AS
>
> Han har nå lagt til følgende:
>
> Det er imidlertid på sin plass å nevne at dette er et tema
> hvor de 'lærde' strides. Dvs. det finnes enkelte som mener at det er mulig
> å lese overskrevne data. Men vi har som sagt ikke kunnet finne noe
> vitenskaplig dokumentasjon eller beskrivelser av hvordan dette kan
> gjøres.
>
> -----------------------------------------------------------------------------
>
>   -jf
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: secure erasure of files?

[Richard B. Johnson]

On Wed, 20 Feb 2002, Jens Schmidt wrote:

> Hi All,
> 
> The information in this message is very interesting, and provided me with
> a real insight in the matter of magnetic media.
> 

You can't read back overwritten magnetic media by running some
magic program. However, it is not impossible to recover some
data from disk-drive tracks that have been overwritten. This
is because new data is unlikely to be written exactly over the
same area of a track as old data. This happens because of the
normal expansion and contraction of the media with changing
temperature.

By changing the alignment phase of the tracking servo, it
may be possible to read data that was written at a different
temperature. The data is not recovered intact, but a significant
percentage may be recovered, enough to be useful in a forensic
investigation or, perhaps espionage.

Since it is not impossible to recover such data, this information
is often exploited. I observed an advertisement on the Web which
showed a burned up disk-drive with its platters exposed and
charred. Implicit in the advertisement was that data had been
recovered from this drive. Such advertising is "legendary".

You can prevent recovery of overwritten data by keeping your
machine running all the while, thus at a near constant temperature.
If you periodically execute a program which writes a file large
enough to fill up the media, sync the file data, then delete it.
The result will be a drive clean enough so it would not contain
evidence of previous file-content except for file names. So,
if the file-name is not evidence you can be assured that the
"Thought Police" will not find your computer useful.

Truly paranoid persons should use file-names like this:

total 2916
drwxr-xr-x  53 root     root         4096 Feb 19 09:06 .
drwxr-xr-x  24 root     root         4096 Feb 19 05:03 ..
-rw-r--r--   1 root     root         1093 Oct 10 13:56 0x00000000 
-rw-r--r--   1 root     root          366 Aug  3  2001 0x00000001 
-rw-r--r--   1 root     root           69 Apr 24  1998 0x00000002 
-rw-r--r--   1 root     root          625 Jun  3  1998 0x00000003 
-rw-r--r--   1 root     root           43 Aug  4  1999 0x00000004 
-rw-r--r--   1 root     root          399 Feb 13 11:16 0x00000005 
-rw-r--r--   1 root     root           69 Aug  4  1999 0x00000006 
drwxr-xr-x   2 root     root         4096 Oct  9  2000 0x00000007 
drwxr-xr-x   2 root     root         4096 Aug 15  2000 0x00000008


Such file names contain no evidence of the file or directory
content. However, you note that there is some information conveyed
in the date-time. The date-time can be useful for evidence because
it (may) establish when the file was created or modified.

You can fix this problem by setting your system time to 1/1/1970
every time you boot. You could also install Windows-2000/Professional.
That Operating System doesn't stay running long enough to provide
useful evidence.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: secure erasure of files?

[Petro]

On Tue, Feb 19, 2002 at 09:24:08AM -0500, Richard B. Johnson wrote:
> On Wed, 20 Feb 2002, Jens Schmidt wrote:
 
> Truly paranoid persons should use file-names like this:

    Truly paranoid persons use an encrypted file system whose code
    they've audited themselves. 




-- 
Share and Enjoy. 

RE: secure erasure of files?

[M. Edward Borasky]

Truly secure people dispose of classified hard drives in vats of aqua regia
:)

M. Edward Borasky
The COUGAR Project

znmeb@borasky-research.net
http://www.borasky-research.com/Cougar.htm

Re: secure erasure of files?

[Holger Lubitz]

"M. Edward Borasky" proclaimed:
> 
> Truly secure people dispose of classified hard drives in vats of aqua regia
> :)

One could also use a blast furnace, I'd guess ;)

Holger

Re: secure erasure of files?

[Petro]

On Wed, Feb 20, 2002 at 07:20:13PM -0800, M. Edward Borasky wrote:
> Truly secure people dispose of classified hard drives in vats of aqua regia
> :)

    I was thinking thermite, but I've always been a fan of pyrotechnics. 

-- 
Share and Enjoy. 

Re: secure erasure of files?

[Jesse Pollard]

---------  Received message begins Here  ---------

> 
> >I would strongly encourage somebody with fluent Norsk/English skills
> >to do a translation and post it to the list.
> 
> I'll do my very best ...
> 
> (translated by Roy Sigurd Karlsbakk - please don't spam me in case of bad
> speling :)
> 
> With permission from the leader of Research and Deveopment department, I
> quote his complete answer:
> 
> I'll try to answer your questions:
> 
> The short answer is: No. It is not possible to read data that are (really)
> physically overwritten.

[snip]

In the non-destructive read case - true.

HOWEVER: forensic specialists can:

	http://www.cs.auckland.ac.nz/~pgut001/secure_del.html
or (same paper)
	http://www.usenix.org/publications/library/proceedings/sec96/full_papers/gutmann/

> 
> Addition:
> 
> Still, it should be said that this is being argued upon between the
> 'wise' ones. This is - there are people that mean it is possible
> to read/recover overwritten data. But we have, as mentioned above,
> not found any scientific documentation or decriptions of how this
> can be done.

See the paper referenced above. There may be more recent documents, but
this one is quite clear on the limitations of erasure using the standard
drive electronics.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

Re: secure erasure of files?

[Roy Sigurd Karlsbakk]

>I would strongly encourage somebody with fluent Norsk/English skills
>to do a translation and post it to the list.

I'll do my very best ...

(translated by Roy Sigurd Karlsbakk - please don't spam me in case of bad
speling :)

With permission from the leader of Research and Deveopment department, I
quote his complete answer:

I'll try to answer your questions:

The short answer is: No. It is not possible to read data that are (really)
physically overwritten.

Still, the reason to this is is a little different than what you are describing.
To speak reasonably about this, we firstly need some basic understanding of how
data is stored on a harddisk. A harddisk does not manipulate individual bits, but
flux change. Simply explained is 'flux direction' however the magnetic field points
clockwise or counter-clockwise. Thus, a 'flux change' is where the flux changes from
CCW to CW or the other way around. The mapping between flux changes is not
one-to-one.
This means that we DO NOT use CW=0, CCW=1, but rather have each flux
change
giving the origin pf 2.5 to 3 bits in addition to the disks sequence detection.
This means that it does not attempt to detect each bit isolated, but rather decodes
a sequence at a time (typically 4096 bit = 1 sector).

This sequence detection is quite the same as a human reading a bad
fax. If we try to read the fax letter by letter, we may for instance
mistake an 'a' with an 'o'. If this letter is part of the word 'bank',
and we read it letter by letter, we'll end up with 'bonk. However, if we
look at the whole word (the sequence of letters), we can probably see
the most probable word is 'bank'.

After data is overwritten, we can measure how strong the (field of the)
old data is compared to the new ones. This means that all 'old' signals
never really disappear. Still, our investigation shows that there is no
officially documented way of how to change these (old) signals back to
the origial data-

It may seem this will demand trail-breaking discoveries in many different
fields: Non-linear analysis and modelling, low-noice electronics (cryo-
electronics), computer technology (super-fast number-chrunchers)

This was the long (complicated) answer :)

What is sure: Ibas does not know any documented methods, scientific
environments or commercial services that do or demonstrate reading
of overwritten data.

--
Thor Arne Johansen
Avdelingssjef FoU, Ibas AS

Addition:

Still, it should be said that this is being argued upon between the
'wise' ones. This is - there are people that mean it is possible
to read/recover overwritten data. But we have, as mentioned above,
not found any scientific documentation or decriptions of how this
can be done.

-----------------------------------------------------------------------------
--
--
Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA

Computers are like air conditioners.
They stop working when you open Windows.

Re: secure erasure of files?

[Rogier Wolff]

Roy Sigurd Karlsbakk wrote:
> What is sure: Ibas does not know any documented methods, scientific
> environments or commercial services that do or demonstrate reading
> of overwritten data.

Ah. Our data-recovery competitor IBAS. Short addition: "Same here". 


Wietze Venema, (I'm sure you know what he's famous for) once had a
slide in his show that showed an image of magnetic track that had been
overwritten with new data. You could visually see that the head had
been mis-positioned by about 0.1 track-width, such that the old data
was still visible from below. This would lead you to believe that
possibly your old data could be retrieved. 

This is easier said than done. In this case they had been able to
image the patter at over 100 times more resolution than the magnetic
head of the drive. Imaging a whole 10G platter this way would yield
you a few terabytes of data, from which you can try to find the old
data back. Good luck.

Some success is rumored to be able to be achieved by sampling the
normal signal, and then subtracting the "expected signal assuming the
current sequence of bits that was read". That way you might be able to
recover the information that peeks out from below. On the other hand,
the electronics is ALREADY partly doing this to recover the normal
data that is read from the platter...

In practise all this doesn't work: The head will not be mispositioned
0.1 track to the same side during the whole revolution. Thus you will
have parts of the previous data generation peeking out on the left
side for part of the track and data from the generation before on the
other side. Which you will see is not predetermined.

Now, if there are military secrets on the drive, the opponent may want
to acutally go and image the platter, and reconstruct the data on a
bit by bit basis. If you then happen to stumble on just a couple of
"launch codes" or something like that, then "all hell breaks loose". 
Thus the chance that this might possibly happen is to be prevented. 


				Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 

Re: secure erasure of files?

[Martin J. Bligh]

> Some success is rumored to be able to be achieved by sampling the
> normal signal, and then subtracting the "expected signal assuming the
> current sequence of bits that was read". That way you might be able to
> recover the information that peeks out from below. 

It's more than rumour - I've seen this done. Dr Solomon's (whatever
they called their data recovery branch), early 1990's, England.
Maybe it was easier on older hardware like the MFM / RLL disks,
and certainly easier to piece together fragmented data with earlier
file formats.

I believe the point of overwriting 3 times (or whatever) is to reduce
the "subtracted difference" to noise levels where it's no longer useful.

> In practise all this doesn't work: The head will not be mispositioned
> 0.1 track to the same side during the whole revolution. Thus you will
> have parts of the previous data generation peeking out on the left
> side for part of the track and data from the generation before on the
> other side. Which you will see is not predetermined.

This only deals with your first method (which I agree, sounds unlikely
to work).

M.

PS. I've also seen a disk arm being wound across an opened disk
platter by a micrometer strapped to the head by a rubber band, 
to recover real data. Most amusing ;-).

Re: secure erasure of files?

[Rogier Wolff]

Martin J. Bligh wrote:
> > Some success is rumored to be able to be achieved by sampling the
> > normal signal, and then subtracting the "expected signal assuming the
> > current sequence of bits that was read". That way you might be able to
> > recover the information that peeks out from below. 
> 
> It's more than rumour - I've seen this done. Dr Solomon's (whatever
> they called their data recovery branch), early 1990's, England.
> Maybe it was easier on older hardware like the MFM / RLL disks,
> and certainly easier to piece together fragmented data with earlier
> file formats.

Maybe the difference is in "what's the goal". For datarecovery we
don't really care about just a couple of bits here and there: We want
to piece together the whole thing. 

If you don't want a piece of your data getting into wrong hands
however, you'd better be safe than sorry.

So I (and the Ibas guy) are talking about practical recovery of a
useful amount of data, while even a couple of bits is in theory
dangerous if you really want it "gone".

> I believe the point of overwriting 3 times (or whatever) is to reduce
> the "subtracted difference" to noise levels where it's no longer useful.

Right. 

		Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 

Re: secure erasure of files?

[Andreas Dilger]

On Feb 19, 2002  19:48 +0100, Rogier Wolff wrote:
> Maybe the difference is in "what's the goal". For datarecovery we
> don't really care about just a couple of bits here and there: We want
> to piece together the whole thing. 
> 
> If you don't want a piece of your data getting into wrong hands
> however, you'd better be safe than sorry.
> 
> So I (and the Ibas guy) are talking about practical recovery of a
> useful amount of data, while even a couple of bits is in theory
> dangerous if you really want it "gone".

So, as others have said, if your data is so important that you are
worried about people taking the platter and putting it under a
scanning-tunneling microscope (or whatever is in vogue) to recover
deleted data, then you should be one million times as worried about
undeleted data on the same disk (i.e. what happens if they steal or
copy the disk _before_ you delete this precious data).

The net result is that this data should never hit the disk unencrypted
in the first place, at which point you don't need to worry about the
deletion step.  You have encrypted swap and encrypted loopback filesystems,
and you have proper procedure to ensure the keys are safe, and all is well.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: secure erasure of files?

[Roy Sigurd Karlsbakk]

There is one point, though.

Ibas told me, at at briefing, that they had recovered data, overwritten 37
times. This was in '93.

This also might tell the text underneath is false.

anyone?

roy

On Tue, 19 Feb 2002, Roy Sigurd Karlsbakk wrote:

> >I would strongly encourage somebody with fluent Norsk/English skills
> >to do a translation and post it to the list.
>
> I'll do my very best ...
>
> (translated by Roy Sigurd Karlsbakk - please don't spam me in case of bad
> speling :)
>
> With permission from the leader of Research and Deveopment department, I
> quote his complete answer:
>
> I'll try to answer your questions:
>
> The short answer is: No. It is not possible to read data that are (really)
> physically overwritten.
>
> Still, the reason to this is is a little different than what you are describing.
> To speak reasonably about this, we firstly need some basic understanding of how
> data is stored on a harddisk. A harddisk does not manipulate individual bits, but
> flux change. Simply explained is 'flux direction' however the magnetic field points
> clockwise or counter-clockwise. Thus, a 'flux change' is where the flux changes from
> CCW to CW or the other way around. The mapping between flux changes is not
> one-to-one.
> This means that we DO NOT use CW=0, CCW=1, but rather have each flux
> change
> giving the origin pf 2.5 to 3 bits in addition to the disks sequence detection.
> This means that it does not attempt to detect each bit isolated, but rather decodes
> a sequence at a time (typically 4096 bit = 1 sector).
>
> This sequence detection is quite the same as a human reading a bad
> fax. If we try to read the fax letter by letter, we may for instance
> mistake an 'a' with an 'o'. If this letter is part of the word 'bank',
> and we read it letter by letter, we'll end up with 'bonk. However, if we
> look at the whole word (the sequence of letters), we can probably see
> the most probable word is 'bank'.
>
> After data is overwritten, we can measure how strong the (field of the)
> old data is compared to the new ones. This means that all 'old' signals
> never really disappear. Still, our investigation shows that there is no
> officially documented way of how to change these (old) signals back to
> the origial data-
>
> It may seem this will demand trail-breaking discoveries in many different
> fields: Non-linear analysis and modelling, low-noice electronics (cryo-
> electronics), computer technology (super-fast number-chrunchers)
>
> This was the long (complicated) answer :)
>
> What is sure: Ibas does not know any documented methods, scientific
> environments or commercial services that do or demonstrate reading
> of overwritten data.
>
> --
> Thor Arne Johansen
> Avdelingssjef FoU, Ibas AS
>
> Addition:
>
> Still, it should be said that this is being argued upon between the
> 'wise' ones. This is - there are people that mean it is possible
> to read/recover overwritten data. But we have, as mentioned above,
> not found any scientific documentation or decriptions of how this
> can be done.
>
> -----------------------------------------------------------------------------
> --
> --
> Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA
>
> Computers are like air conditioners.
> They stop working when you open Windows.
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

--
Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA

Computers are like air conditioners.
They stop working when you open Windows.

RE: secure erasure of files?

[Torrey Hoffman]

IIRC, last time this topic came up, the best answer was:

Given modern disk hardware (block remapping) and operating 
system behavior, no, there is no way to securely delete files 
regardless of OS or filesystem.  

(AFAIK, the transparent hardware block remapping cannot be
detected or worked around in software - any software - but 
perhaps the IDE experts here know otherwise. )

If you don't want a sufficiently determined attacker to be
able to read your data from the disk, don't write it.

The solution is to use encryption and make sure your data is 
never, ever written to disk unencrypted.  In particular, use 
encrypted swap and encrypted loopback filesystems.

Torrey

secure erasure of files?

[Roy Sigurd Karlsbakk]

hi all

Does anyone know if it'll be hard to enable a <em>secure</em> deletion of
files? What I mean is not merely overwriting it with NULLs, but rather
using a more sophisticated overwrite, like the IBAS ExpertEraser software
(http://www.ibas.com/erasure/)

Is this hard/possible/doable?

roy

--
Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA

Computers are like air conditioners.
They stop working when you open Windows.

Re: secure erasure of files?

[Davidovac Zoran]

there is srm (secure rm) somewhere on the net
here srm.sourceforge.net
srm - secure file deletion for posix systems

happy srm,

      Zoran

   srm is a secure replacement for rm(1). Unlike the standard rm, it
   overwrites the data in the target files before unlinking them. This
   prevents command-line recovery of the data by examining the raw block
   device. It may also help frustrate physical examination of the disk,
   although it's unlikely that it can completely prevent that type of
   recovery. It is, essentially, a paper shredder for sensitive files.

   srm is ideal for personal computers or workstations with Internet
   connections. It can help prevent malicious users from breaking in and
   undeleting personal files, such as old emails. It's also useful for
   permanently removing files from expensive media. For example, cleaning
   your diary off the zip disk you're using to send vacation pictures to
   Uncle Lou. Because it uses the exact same options as rm(1), srm is
   simple to use. Just subsitute it for rm whenever you want to destroy
   files, rather than just unlinking them.


On Tue, 12 Feb 2002, Roy Sigurd Karlsbakk wrote:

> Date: Tue, 12 Feb 2002 14:12:49 +0100 (CET)
> From: Roy Sigurd Karlsbakk <roy@karlsbakk.net>
> To: linux-kernel@vger.kernel.org
> Subject: secure erasure of files?
>
> hi all
>
> Does anyone know if it'll be hard to enable a <em>secure</em> deletion of
> files? What I mean is not merely overwriting it with NULLs, but rather
> using a more sophisticated overwrite, like the IBAS ExpertEraser software
> (http://www.ibas.com/erasure/)
>
> Is this hard/possible/doable?
>
> roy
>
> --
> Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA
>
> Computers are like air conditioners.
> They stop working when you open Windows.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: secure erasure of files?

[Padraig Brady]

And a patch to call it automatically:
http://www.tpgn.net/exploits/linux/secrm.kernel.patch.2.2.14.txt

On a related note is there any way to clear the unused portion
of the last block of a file. Obviously it's file system dependent,
but what happens in general on ext2?

Padraig.

Davidovac Zoran wrote:
> there is srm (secure rm) somewhere on the net
> here srm.sourceforge.net
> srm - secure file deletion for posix systems
> 
> happy srm,
> 
>       Zoran
> 
>    srm is a secure replacement for rm(1). Unlike the standard rm, it
>    overwrites the data in the target files before unlinking them. This
>    prevents command-line recovery of the data by examining the raw block
>    device. It may also help frustrate physical examination of the disk,
>    although it's unlikely that it can completely prevent that type of
>    recovery. It is, essentially, a paper shredder for sensitive files.
> 
>    srm is ideal for personal computers or workstations with Internet
>    connections. It can help prevent malicious users from breaking in and
>    undeleting personal files, such as old emails. It's also useful for
>    permanently removing files from expensive media. For example, cleaning
>    your diary off the zip disk you're using to send vacation pictures to
>    Uncle Lou. Because it uses the exact same options as rm(1), srm is
>    simple to use. Just subsitute it for rm whenever you want to destroy
>    files, rather than just unlinking them.
> 
> 
> On Tue, 12 Feb 2002, Roy Sigurd Karlsbakk wrote:
> 
> 
>>Date: Tue, 12 Feb 2002 14:12:49 +0100 (CET)
>>From: Roy Sigurd Karlsbakk <roy@karlsbakk.net>
>>To: linux-kernel@vger.kernel.org
>>Subject: secure erasure of files?
>>
>>hi all
>>
>>Does anyone know if it'll be hard to enable a <em>secure</em> deletion of
>>files? What I mean is not merely overwriting it with NULLs, but rather
>>using a more sophisticated overwrite, like the IBAS ExpertEraser software
>>(http://www.ibas.com/erasure/)
>>
>>Is this hard/possible/doable?
>>
>>roy

Re: secure erasure of files?

[Andreas Ferber]

On Tue, Feb 12, 2002 at 02:41:19PM +0100, Davidovac Zoran wrote:

(srm maintainer cc'ed)

> there is srm (secure rm) somewhere on the net
> here srm.sourceforge.net
> srm - secure file deletion for posix systems

Broken as designed, it simply /can't/ work reliably (not to mention
the other comments that you can even recover data overwritten multiple
times).

Nothing stops the kernel (or the filesystem for that matter) from
shuffling around disk blocks while you are overwriting the file. You
may end up overwriting other disk blocks than the data you want to
hide lives in if the filesystem decides that your file may fit better
into other blocks, which leaves the original data completely intact.

I don't know if any filesystem currently relocates blocks if you
overwrite a file, but it's certainly possible and allowed (everything
else except the filesystem itself simply must not care where the data
actually ends up on the disk).

In addition to the design breakage, the current implementation of srm
is simply crap. Here is the part actually overwriting the file:

---------- snip ----------
  int i = 0;

  lseek(file, 0, SEEK_SET);
  while (i < file_size - buffsize)
    i += write(file, buffer, buffsize);
  write(file, buffer, file_size - i);
---------- snip ----------

Guess what happens if you try to srm a file longer than INT_MAX bytes?
And what if write() returns an error? Ghee, overwriting the file
backwards from the beginning, until infinity? Impressive...

Ah, and look at rename_unlink(): it tries to "overwrite" the
original filename with random characters. It does this by generating a
new 14 character random filename and rename()ing the file to that name
(keeping it in the same directory). Well, for example on plain stupid
(no pun ;-) ext2fs there are /many/ situations where the new directory
entry ends up in a totally different place than the old one (e.g. a
filename shorter than 14 characters if there is no free space around
the old direntry).

Well, enough for now...

Andreas
-- 
       Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
     ---------------------------------------------------------
         +49 521 1365800 - af@devcon.net - www.devcon.net

Re: secure erasure of files?

[Jan Hudec]

> I don't know if any filesystem currently relocates blocks if you
> overwrite a file, but it's certainly possible and allowed (everything
> else except the filesystem itself simply must not care where the data
> actually ends up on the disk).

AFAIK, ext2 tries to defragment files when possible. Thus if the file was
fragmented and the blocks after some fragment are free, it will use these
instead of the original ones somewhere far apart.

--------------------------------------------------------------------------------
                  				- Jan Hudec `Bulb' <bulb@ucw.cz>

Re: secure erasure of files?

[Andrew Morton]

Jan Hudec wrote:
> 
> > I don't know if any filesystem currently relocates blocks if you
> > overwrite a file, but it's certainly possible and allowed (everything
> > else except the filesystem itself simply must not care where the data
> > actually ends up on the disk).
> 
> AFAIK, ext2 tries to defragment files when possible.

We wish.

> Thus if the file was
> fragmented and the blocks after some fragment are free, it will use these
> instead of the original ones somewhere far apart.

Nope - if you're overwriting a part of the file which already has
allocated blocks, ext2/3 will use the same blocks.

Of course, the disk itself may decide to go and remap the block
if it decides that part of the media is getting tired.

-

Re: secure erasure of files?

[Jeff Garzik]

Andrew Morton wrote:
> 
> Jan Hudec wrote:
> >
> > > I don't know if any filesystem currently relocates blocks if you
> > > overwrite a file, but it's certainly possible and allowed (everything
> > > else except the filesystem itself simply must not care where the data
> > > actually ends up on the disk).
> >
> > AFAIK, ext2 tries to defragment files when possible.
> 
> We wish.

ext2meta will defragment yer files for ya, though.  Coming soon to a
kernel near you :)

My home directory disk, a raid-1 mirror, just crossed 15 percent
fragmentation last week, I noticed...  The filesystem is maybe 1.5-2
years old.

	Jeff


-- 
Jeff Garzik      | "I went through my candy like hot oatmeal
Building 1024    |  through an internally-buttered weasel."
MandrakeSoft     |             - goats.com

Re: secure erasure of files?

[Helge Hafting]

Andreas Ferber wrote:

> I don't know if any filesystem currently relocates blocks if you
> overwrite a file, but it's certainly possible and allowed (everything
> else except the filesystem itself simply must not care where the data
> actually ends up on the disk).
> 
A log-structured fs will write new blocks everytime, afaik.
Ext3 with data journalling keeps copies of recently written data
in the journal.  Now, if you create a "secret" file and then overwrite
it you'll still find a copy in the journal until the journal wraps
It may not wrap if the next thing you do is umount/shutdown.

A secure rm must know the fs it works with.  A better solution
is to overwrite the entire partition with garbage. The only
perfect way is to destroy the magnetic surfaces though.


Helge Hafting

Re: secure erasure of files?

[Mike Fedyk]

On Wed, Feb 13, 2002 at 10:33:33AM +0100, Helge Hafting wrote:
> Andreas Ferber wrote:
> 
> > I don't know if any filesystem currently relocates blocks if you
> > overwrite a file, but it's certainly possible and allowed (everything
> > else except the filesystem itself simply must not care where the data
> > actually ends up on the disk).
> > 
> A log-structured fs will write new blocks everytime, afaik.

Ext3 only does that on truncate in ordered/writeback mode and probably in
data journaling mode too.

> Ext3 with data journalling keeps copies of recently written data
> in the journal.  Now, if you create a "secret" file and then overwrite
> it you'll still find a copy in the journal until the journal wraps
> It may not wrap if the next thing you do is umount/shutdown.
>
> A secure rm must know the fs it works with.  A better solution
> is to overwrite the entire partition with garbage. The only
> perfect way is to destroy the magnetic surfaces though.
>

Yep, very true.

Phycally breaking the drive apart into pieces and then burning should
suffice. 

Mike

Re: secure erasure of files?

[Tom Vier]

On Tue, Feb 12, 2002 at 02:12:49PM +0100, Roy Sigurd Karlsbakk wrote:
> Does anyone know if it'll be hard to enable a <em>secure</em> deletion of
<snip>
> Is this hard/possible/doable?

read:

http://wipe.sf.net/

it's the site of my file wiper. note the list of situations in which it
definetly will not work. if you really don't want plaintext laying around,
then using an encrypted fs is the only way.

a new version will be out soon, with a couple minor bug fixes. wipe is
mostly useful for wiping a drive you're selling of personal stuff. 

also, don't forget about swap. if, for example, a proc reads the contents of
a file from an encrypted fs into a buffer, the buffer can be written to swap
in plaintext.

-- 
Tom Vier <tmv5@home.com>
DSA Key id 0x27371A2C

Re: secure erasure of files?

[Jeff Garzik]

Tom Vier wrote:
> On Tue, Feb 12, 2002 at 02:12:49PM +0100, Roy Sigurd Karlsbakk wrote:
> > Does anyone know if it'll be hard to enable a <em>secure</em> deletion of
> <snip>
> > Is this hard/possible/doable?
> 
> read:
> 
> http://wipe.sf.net/

The program /usr/bin/shred has existing in GNU fileutils, and thus most
Linux systems, for years and years...

	Jeff



-- 
Jeff Garzik      | "I went through my candy like hot oatmeal
Building 1024    |  through an internally-buttered weasel."
MandrakeSoft     |             - goats.com

Re: secure erasure of files?

[Bill Davidsen]

On Tue, 12 Feb 2002, Roy Sigurd Karlsbakk wrote:

> Does anyone know if it'll be hard to enable a <em>secure</em> deletion of
> files? What I mean is not merely overwriting it with NULLs, but rather
> using a more sophisticated overwrite, like the IBAS ExpertEraser software
> (http://www.ibas.com/erasure/)
> 
> Is this hard/possible/doable?

If you want to prevent casual reads with user tools which don't physically
destroy the hardware, a single overwrite is probablt fine. If you think
someone would want the info enough to take the drive apart, encrypt
everything including swap and tmp, using DES3 or similar extremely hard
code.

You have to know your exposure, who will try to recover the data and how
hard they will try.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.