reiser4 mkfs, cryptcompress, tail packing, etc.

reiser4 mkfs, cryptcompress, tail packing, etc.

[Gordan Bobic]

Hi,

It's been a while since I last tried reiser4, and having just looked at 
the wiki, I couldn't find any info at all on the cryptocompress plugin 
and tail packing options at mkfs time specifically relating to reiser4. 
Also, is there a nolog option on reiser4 as there is on reiserfs?

The reason I ask is because I am looking for a file system that improves 
on ext2 for very slow writing flash media (think cheap USB sticks, SD 
cards, etc.). So, ideally, I want to disable journalling and enable 
compression, to minimize the number of writing to the underlying media. 
In theory, compression with journalling switched off should give reiser4 
an edge in this use-case, so I just wanted to check what the current 
state of the compression is (couldn't find any references to -o 
create=ccreg40 in the wiki), and that it works as one might expect 
(compressing before committing to disk).

Another question (not sure if it is specifically related to reiser4 or 
generic) is about compressed hard-linked DLLs and mmap. Specifically, if 
a .so is hard-linked in two places, dynamically linking to each instance 
causes both to be mmapped to the same memory since they'll have the same 
inode, and that means the memory is only used once. How does this work 
if the .so is compressed? Does it all still work the same, with the 
decompressed file being in a single mmap for both dynamically linked 
instances?

Many thanks.

Gordan

Re: reiser4 mkfs, cryptcompress, tail packing, etc.

[Edward Shishkin]

Gordan Bobic wrote:
> Hi,

Hello.

>
> It's been a while since I last tried reiser4, and having just looked 
> at the wiki, I couldn't find any info at all on the cryptocompress 
> plugin and tail packing options at mkfs time specifically relating to 
> reiser4.

mkfs.reiser4 -l:

compression modes (works "per file"):

"latt"              (id:0x1 type:0xd) ['Check on dynamic lattice' 
compression mode plugin.]
If logical cluster in incompressible, then switch compression off and 
check every
K-th cluster, where K dynamically increased to 32 on every non-success after
compression. If cluster is compressible, than K is set back to 1, etc..

"ultim"             (id:0x2 type:0xd) ['Check ultimately' compression 
mode plugin.]
Switch compression off forever after the first case of non-success.

"force"             (id:0x3 type:0xd) ['Compress evrything' compression 
mode plugin.]
Force to compress everything.

"conv"              (id:0x4 type:0xd) ['Convert to extent' compression 
mode plugin.]
If the first logical cluster of the file is incompressible, then switch 
compression off forever.
This is default mode, but it works not good on various media-files..

> Also, is there a nolog option on reiser4 as there is on reiserfs?

no sorry, currently reiser4 transaction manager can not be switched off.

>
> The reason I ask is because I am looking for a file system that 
> improves on ext2 for very slow writing flash media (think cheap USB 
> sticks, SD cards, etc.). So, ideally, I want to disable journalling 
> and enable compression, to minimize the number of writing to the 
> underlying media. In theory, compression with journalling switched off 
> should give reiser4 an edge in this use-case, so I just wanted to 
> check what the current state of the compression is (couldn't find any 
> references to -o create=ccreg40 in the wiki), and that it works as one 
> might expect (compressing before committing to disk).

Works stable as expected (i.e. compression "on commits")

>
>
> Another question (not sure if it is specifically related to reiser4 or 
> generic) is about compressed hard-linked DLLs and mmap. Specifically, 
> if a .so is hard-linked in two places, dynamically linking to each 
> instance causes both to be mmapped to the same memory since they'll 
> have the same inode, and that means the memory is only used once. How 
> does this work if the .so is compressed? Does it all still work the 
> same, with the decompressed file being in a single mmap for both 
> dynamically linked instances?

Works as usual: when populating address space by readpage(s) data are 
read from disk and decompressed.
If  page gets dirty, then its data will be compressed in flush time and 
written to disk.
 
>
> Many thanks.

You are welcome,
Edward.

Re: reiser4 mkfs, cryptcompress, tail packing, etc.

[Edward Shishkin]

Edward Shishkin wrote:
> Gordan Bobic wrote:
>> Hi,
>
> Hello.
>
>>
>> It's been a while since I last tried reiser4, and having just looked 
>> at the wiki, I couldn't find any info at all on the cryptocompress 
>> plugin and tail packing options

Reiser4 tail packing policy can be set by "-o formatting=xxx" option.
List of all available options can be displayed by mkfs.reiser4 -l:

"extents"           (id:0x0 type:0x5) ['Extents only' tail policy plugin.]
"tails"             (id:0x1 type:0x5) ['Tails only' tail policy plugin.]
"smart"             (id:0x2 type:0x5) [Smart tail policy: If file is 
smaller than 20K, store it in tails,
otherwise convert it to extents. Default option.

Re: reiser4 mkfs, cryptcompress, tail packing, etc.

[Gordan Bobic]

On 08/24/2010 06:47 PM, Edward Shishkin wrote:

>> Also, is there a nolog option on reiser4 as there is on reiserfs?
>
> no sorry, currently reiser4 transaction manager can not be switched off.

Is there a way to use an external journal and point it at a different 
device (e.g. ramdisk/tmpfs), and force fsck on unclean shutdown instead?

>> Another question (not sure if it is specifically related to reiser4 or
>> generic) is about compressed hard-linked DLLs and mmap. Specifically,
>> if a .so is hard-linked in two places, dynamically linking to each
>> instance causes both to be mmapped to the same memory since they'll
>> have the same inode, and that means the memory is only used once. How
>> does this work if the .so is compressed? Does it all still work the
>> same, with the decompressed file being in a single mmap for both
>> dynamically linked instances?
>
> Works as usual: when populating address space by readpage(s) data are
> read from disk and decompressed.
> If page gets dirty, then its data will be compressed in flush time and
> written to disk.

What about the hard-link issue I mentioned? If two hard-links point to 
the same inode with a compressed shared library, will it work as 
expected and only map the file into shared memory once even if both .so 
files are dyna-linked by different running binaries?

Gordan

Re: reiser4 mkfs, cryptcompress, tail packing, etc.

[Edward Shishkin]

Edward Shishkin wrote:
> Gordan Bobic wrote:
>> Hi,
>
> Hello.
>
>>
>> It's been a while since I last tried reiser4, and having just looked 
>> at the wiki, I couldn't find any info at all on the cryptocompress 
>> plugin and tail packing options at mkfs time specifically relating to 
>> reiser4.
>
> mkfs.reiser4 -l:
>
> compression modes (works "per file"):
>
> "latt"              (id:0x1 type:0xd) ['Check on dynamic lattice' 
> compression mode plugin.]
> If logical cluster in incompressible, then switch compression off and 
> check every
> K-th cluster, where K dynamically increased to 32 on every non-success 
> after
> compression. If cluster is compressible, than K is set back to 1, etc..
>
> "ultim"             (id:0x2 type:0xd) ['Check ultimately' compression 
> mode plugin.]
> Switch compression off forever after the first case of non-success.
>
> "force"             (id:0x3 type:0xd) ['Compress evrything' 
> compression mode plugin.]
> Force to compress everything.
>
> "conv"              (id:0x4 type:0xd) ['Convert to extent' compression 
> mode plugin.]
> If the first logical cluster of the file is incompressible, then 
> switch compression off forever.

Sorry, this is not everything. Here should also go "otherwise switch to 
"latt" mode
(see above)".
So not everything is bad for media-files in this mode.

Re: reiser4 mkfs, cryptcompress, tail packing, etc.

[Edward Shishkin]

Gordan Bobic wrote:
> On 08/24/2010 06:47 PM, Edward Shishkin wrote:
>
>>> Also, is there a nolog option on reiser4 as there is on reiserfs?
>>
>> no sorry, currently reiser4 transaction manager can not be switched off.
>
> Is there a way to use an external journal and point it at a different 
> device (e.g. ramdisk/tmpfs), and force fsck on unclean shutdown instead?

Nop, transaction manager in reiser4 is not a pure journalling.
This is a symbiosis of journalling and copy-on-write technologies,
so journal relocation wouldn't be a correct option here.

There is a possibility to add various transaction plugins to reiser4,
including pure journalling (like in reiserfs) and pure COW (like in btrfs).
However, it would be a serious work, which requires many human hours..

>
>
>>> Another question (not sure if it is specifically related to reiser4 or
>>> generic) is about compressed hard-linked DLLs and mmap. Specifically,
>>> if a .so is hard-linked in two places, dynamically linking to each
>>> instance causes both to be mmapped to the same memory since they'll
>>> have the same inode, and that means the memory is only used once. How
>>> does this work if the .so is compressed? Does it all still work the
>>> same, with the decompressed file being in a single mmap for both
>>> dynamically linked instances?
>>
>> Works as usual: when populating address space by readpage(s) data are
>> read from disk and decompressed.
>> If page gets dirty, then its data will be compressed in flush time and
>> written to disk.
>
> What about the hard-link issue I mentioned? If two hard-links point to 
> the same inode with a compressed shared library, will it work as 
> expected and only map the file into shared memory once even if both 
> .so files are dyna-linked by different running binaries?

Will work as expected.

Edward.