Re: Reiser4 und LZO compression

[PFC <lists@peufeu.com>]

> Maybe, but Reiser4 is supposed to be a general purpose filesystem
> talking about its advantages/disadvantages wrt. gaming makes sense,

	I don't see a lot of gamers using Linux ;)
	But yes, gaming is what pushes hardware development these days, at least  
on the desktop.

	Also, as you said, gamers (like many others) reinvent filesystems and  
generally use the Big Zip File paradigm, which is not that stupid for a  
read only FS (if you cache all file offsets, reading can be pretty fast).  
However when you start storing ogg-compressed sound and JPEG images inside  
a zip file, it starts to stink.

	***************************

> Does the CPU power necessary to do the compression cost more or less  
> than another drive?

	***************************

	It depends, you have to consider several distinct scenarios.
	For instance, on a big Postgres database server, the rule is to have as  
many spindles as you can.
	- If you are doing a lot of full table scans (like data mining etc), more  
spindles means reads can be parallelized ; of course this will mean more  
data will have to be decompressed.
	- If you are doing a lot of little transactions (web sites), it means  
seeks can be distributed around the various disks. In this case  
compression would be a big win because there is free CPU to use ; besides,  
it would virtually double the RAM cache size.

	You have to ponder cost (in CPU $) of compression versus the cost in  
"virtual RAM" saved for caching and the cost in disks not bought.

	***************************

> Do the two processors have separate caches, and thus being overly fined
> grained makes you memory transfer bound or?

	It depends on which dual core system you use ; future systems (like Core)  
will definitely share cache as this is the best option.

	***************************

	If we analyze the results of my little compression benchmarks, we find  
that :
	- gzip is way too slow.
	- lzo and lzf are pretty close.

	LZF is faster than LZO (especially on decompression) but compresses worse.
	So, when we are disk-bound, LZF will be slower.
	When we are CPU-bound, LZF will be faster.

	The differences are not that huge, though, so it might be worthwile to  
weight this against the respective code cleanliness, of which I have no  
idea.

	However my compression benchmarks mean nothing because I'm compressing  
whole files whereas reiser4 will be compressing little blocks of files. We  
must therefore evaluate the performance of compressors on little blocks,  
which is very different from 300 megabytes files.
	For instance, the setup time of the compressor will be important (wether  
some huffman table needs to be constructed etc), and the compression  
ratios will be worse.

	Let's redo a benchmark then.
	For that I need to know if a compression block in reiser4 will be either :
	- a FS block containing several files (ie. a block will contain several  
small files)
	- a part of a file (ie. a small file will be 1 block)

	I think it's the second option, right ?