% of filesystem using tails/tail unpacking (shred related)

% of filesystem using tails/tail unpacking (shred related)

[Brett Russ]

Is there a way to analyze an existing ReiserFS and find out how many of the
blocks are tail packed as opposed to full/partially full from the same file?
This could lead to an impact study of converting it to an unpacked state.

From the information I have received so far, the only way shredding can
reliably work on Reiser, tail packing and data journaling need to be
disabled.  However, existing filesystems using tail packing can not be
simply unpacked to separate blocks without considering the following:

1) as mentioned first, an impact study should be done to determine if
unpacking will overflow the available space on the device

2) an unpack routine would need to be written if it doesn't exist.  With my
current knowledge of Reiser and my timeframe (3 weeks) this is not possible
for me to do IMO.

3) the unpack operation would need to treat the previously packed blocks as
sensitive by shredding them after all tails were relocated but before
returning the block to a free list

4) this operation, depending on the outcome of step 1, would be very
expensive in performance

That's all I can think of right now.  Does this sound accurate?

thanks,
brett

Re: % of filesystem using tails/tail unpacking (shred related)

[Nikita Danilov]

Brett Russ writes:
 > Is there a way to analyze an existing ReiserFS and find out how many of the
 > blocks are tail packed as opposed to full/partially full from the same file?
 > This could lead to an impact study of converting it to an unpacked state.
 > 
 > >From the information I have received so far, the only way shredding can
 > reliably work on Reiser, tail packing and data journaling need to be
 > disabled.  However, existing filesystems using tail packing can not be
 > simply unpacked to separate blocks without considering the following:
 > 
 > 1) as mentioned first, an impact study should be done to determine if
 > unpacking will overflow the available space on the device

Why cannot you do unpacking just before shredding particular file?

 > 
 > 2) an unpack routine would need to be written if it doesn't exist.  With my
 > current knowledge of Reiser and my timeframe (3 weeks) this is not possible
 > for me to do IMO.

Unpacking can be done through REISERFS_IOC_UNPACK ioctl (see
fs/reiserfs/ioctl.c). One should keep file descriptor open across
ioctl() and shredding for this to work reliably.

 > 
 > 3) the unpack operation would need to treat the previously packed blocks as
 > sensitive by shredding them after all tails were relocated but before
 > returning the block to a free list

ioctl doesn't do this.

 > 
 > 4) this operation, depending on the outcome of step 1, would be very
 > expensive in performance
 > 
 > That's all I can think of right now.  Does this sound accurate?
 > 
 > thanks,
 > brett
 > 

Nikita.

 > 

Re: % of filesystem using tails/tail unpacking (shred related)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

On Mon, 28 Oct 2002 12:55:20 +0300, Nikita Danilov said:
> Brett Russ writes:

>  > 1) as mentioned first, an impact study should be done to determine if
>  > unpacking will overflow the available space on the device
> 
> Why cannot you do unpacking just before shredding particular file?

Unpacking before shredding has a subtle hole in it - since you will unpack
into *different* disk blocks and then shred *those* blocks, you will be
leaving unshredded data in the blocks the file occupied before unpacking.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: % of filesystem using tails/tail unpacking (shred related)

[Brett Russ]

Valdis.Kletnieks@vt.edu wrote:
> On Mon, 28 Oct 2002 12:55:20 +0300, Nikita Danilov said:
> 
>>Brett Russ writes:
> 
> 
>> > 1) as mentioned first, an impact study should be done to determine if
>> > unpacking will overflow the available space on the device
>>
>>Why cannot you do unpacking just before shredding particular file?
> 
> 
> Unpacking before shredding has a subtle hole in it - since you will unpack
> into *different* disk blocks and then shred *those* blocks, you will be
> leaving unshredded data in the blocks the file occupied before unpacking.

Exactly, that was item #3 on my list.  So given what has been said thus 
far, the following seems to be true:

a) shred will work reliably on all new files created on a filesystem 
mounted with tail packing disabled

b) to deal with filesystems already tail packed, shredding knowledge 
would need to be added to the unpack ioctl BUT there is no way to 
determine where else the tails have lived on disk prior to any rebalancing.

Which leads to:

c) there is no reliable way to shred anything that has been tail packed.

Please affirm or correct my conclusions, and thank you all for your help 
thus far.

-Brett

Re: % of filesystem using tails/tail unpacking (shred related)

[Oleg Drokin]

Hello!

On Mon, Oct 28, 2002 at 05:38:57PM -0500, Brett Russ wrote:
> a) shred will work reliably on all new files created on a filesystem 
> mounted with tail packing disabled

Yes. Also in reiserfs case (2.4 kernel in current state) tail is packed
on close, if file created was bigger than 16K (with newe cod it is 
switchable and can be also 4K), tail is never packed for such a file.
Actually the rule is more complex than that, see reiserfs_fs.h,
STORE_TAIL_IN_UNFM

> b) to deal with filesystems already tail packed, shredding knowledge 
> would need to be added to the unpack ioctl BUT there is no way to 
> determine where else the tails have lived on disk prior to any rebalancing.

True. If ever there were any tails at all.
Same stuff applies to a case where file was read into memory,
modified a little bit and then written back to disk, truncating or
removing original file first. On any fs. Lots of editors behave like this
and thus making shred not very effecrive.

> Which leads to:
> c) there is no reliable way to shred anything that has been tail packed.

True. But on reiserfs there certain kind of files (i.e. longer than 16K)
that are never packed.
Also with introduction of data journaling, in data journaling mode the data
removed still will be in journal for some time.

Bye,
    Oleg