Re: Reiser4 documentation & specifically current status of repacker, compression, and semantics

[Edward Shishkin <edward@namesys.com>]

Hans Reiser wrote:

>Edward Shishkin wrote:
>
>  
>
>>Hans Reiser wrote:
>>
>>    
>>
>>>John Gilmore wrote:
>>>
>>>      
>>>
>>>>So the first beta of compression will only have the option to
>>>>control it at mkfs time?
>>>>
>>>>  
>>>>        
>>>>
>>>It should be explained that on the first flush to disk of a new file the
>>>default compress plugin will test to see if the first 64k is
>>>compressable, and if it is not then it converts the file to a regular
>>>file plugin.
>>>
>>> 
>>>
>>>      
>>>
>>a little correction:
>>this is a compression transform plugin that is converted at flush
>>time, not file plugin
>>
>>    
>>
>Please define in more detail please.
>

http://www.namesys.com/cryptcompress_design.html
"Handling incompressible data"
and
http://www.namesys.com/cryptcompress-related_plugins.html
"Compression mode plugin"

>  Particularly, define whether large
>files that cannot be compressed get stored using extents and everything
>else that regular files use.
>
>  
>

Currently files powered by cryptcompress file plugin have its data 
stored only in the
items of CTAIL_ID

>Also, I thought we convert at first flush time, not at every flush
>time.  Did you write it to check at every flush? 
>

no, just turning off and no anymore check, but it can be easily changed

> Not entirely sure we
>want to do that for large multimedia files that are compressed by
>hardware, it could add a bunch of CPU, and I would think it would be
>more complex code, because now you have to convert extents, etc.
>

Yes, such converting seems to be a complicated workaround. We will need
something like "clustered extents"


>  I
>would be concerned that if you always have to check for compression with
>every little write to a file, the cost of that check will be substantial
>for many purposes.
>
>  
>

Since compression is going per cluster, we can consider a set of nominated
clusters as a pattern to check if we need to turn compression off[on] 
for the
whole file. Say, if the clusters of indexes #(N*k), k=0, 1, 2, ...  are
incompressible[compressible], then turn compression off[on].
The smaller N - the more substantial cost of check in the case when 
compression is off.

>Hans
>
>
>  
>