Re: [PATCH] Patch to integrate RapidDisk and RapidCache RAM Drive / Caching modules into the kernel

[Petros Koutoupis <petros@petroskoutoupis.com>]

Christoph (and all),

I hope this message finds you well. The unfortunate truth is, had I the 
courage and confidence to submit this years ago, we wouldn't be having 
this conversation. Anyway, to address your questions:

1.  Unlike the already mainline ramdisk driver, RapidDisk is designed to 
be managed dynamically. That is, instead of configuring a fixed number 
of volumes and volume sizes as compile/boot time variables, RapidDisk 
will allow you to add, remove, and resize your RAM drive(s) at runtime. 
Besides, the built in module is designed to work with smaller sizes in 
mind while RapidDisk focuses on larger sizes that can reach to the 
multiple Gigabytes or even Terabytes. Much like the built in module, it 
will allocate pages as they are needed which allows for over 
provisioning (not that it is advised) of volume sizes.

2. The majority of RapidDisk code focuses on the use of Volatile memory. 
The support for Non-Volatile memory is a bit newer and there may be some 
overlap here with the recently integrated pmem code. The only advantage 
to having this code within RapidDisk is to provide the user with the 
ability to manage both technologies simultaneously, through a single 
interface.

3. The RapidCache component is designed around the Non-Volatile 
functionality of RapidDisk (hence the block-level Write-Through 
caching). It is also coded and optimized around the RapidDisk 
sizes/variables, out-of-box. It is worth noting that I am in the process 
of expanding this module to add deduplication support. This will 
leverage RapidDisk's ability to allocate pages only when needed and 
reduce the cache's memory footprint; making more out of less.

Thoughts, suggestions, and concerns are always welcome.

On 9/28/15 1:49 AM, Christoph Hellwig wrote:
> On Sun, Sep 27, 2015 at 12:17:24PM -0500, Petros Koutoupis wrote:
>> Attached is a patch for two modules: RapidDisk & RapidCache. RapidDisk is a
>> Linux RAM drive module which allows the user to dynamically create, remove,
>> and resize RAM-based block devices. RapidDisk is designed to work with both
>> volatile and non-volatile memory. In the case of volatile memory, memory is
>> allocated only when needed. The RapidCache module in turn utilizes a RapidDisk
>> volume as a FIFO Write-Through caching node to a slower block device.
> Hi Petros,
>
> this is three things at the same time!  We already have a ramdisk
> driver, a pmem drive, bcache and dm-cache, so for each of them please
> explain why we'd want to duplicate them instead of adding whatever
> features you need to them.  First step is to identify those features.