Re: [PATCH] Patch to integrate RapidDisk and RapidCache RAM Drive / Caching modules into the kernel

[Petros Koutoupis <petros@petroskoutoupis.com>]

Christoph and Austin,

You both have provided me with some valuable feedback. I will do what I 
can to clean this patch up and in turn apply the same dynamic 
functionality to the already in-kernel module. Also please see my 
replies below.

On 9/29/15 9:32 AM, Austin S Hemmelgarn wrote:
> On 2015-09-28 12:45, Petros Koutoupis wrote:
>> Christoph,
>>
>> See my replies below....
>>
>> On 9/28/15 11:29 AM, Christoph Hellwig wrote:
>>> Hi Petros,
>>>
>>> On Mon, Sep 28, 2015 at 09:12:13AM -0500, Petros Koutoupis wrote:
>>>> 1.  Unlike the already mainline ramdisk driver, RapidDisk is designed
>>>> to be
>>>> managed dynamically. That is, instead of configuring a fixed number of
>>>> volumes and volume sizes as compile/boot time variables, RapidDisk 
>>>> will
>>>> allow you to add, remove, and resize your RAM drive(s) at runtime.
>>>> Besides,
>>>> the built in module is designed to work with smaller sizes in mind 
>>>> while
>>>> RapidDisk focuses on larger sizes that can reach to the multiple
>>>> Gigabytes
>>>> or even Terabytes. Much like the built in module, it will allocate
>>>> pages as
>>>> they are needed which allows for over provisioning (not that it is
>>>> advised)
>>>> of volume sizes.
>>> The ramdisk driver allows to selects sizes and count at module load
>>> load.  I agree that having runtime control would be even better, but
>>> that's best done by adding a runtime interface to the existing driver
>>> instead of duplicating it.
>> I understand the concern and I will definitely scope out this approach,
>> although at the moment, I am not sure how both approaches will play nice
>> together. As mentioned above, the current implementation requires the
>> predefined number of ram drives with the specified size to be configured
>> at boot time (or compiled into the kernel). The only wiggle room I see
>> for runtime control is resizing individual volumes.
> Just because there is not code currently to do dynamic 
> allocation/freeing of ramdisks in the current driver doesn't mean that 
> it isn't possible, it just means that nobody has written code to do it 
> yet.  This functionality would be extremely useful (I often use 
> ramdisks on a VM host as a small amount of very fast swap space for 
> the virtual machines).  On top of that, the deduplication would be a 
> wonderful feature, although it may already be indirectly implemented 
> through KSM (that is, when KSM is on and configured to scan 
> everything, I'm not sure if it scans memory used by the ramdisks or not).
>
To my understanding KSM is only applied to KVM deployments. One way I 
have seen my caching module work is users/vendors have a block device, 
map it to a RapidDisk RAM drive as a RAM based Write-Through caching 
node and in turn export it via a traditional SAN. The idea behind adding 
deduplication to this module is to minimize the RAM drive footprint when 
used as a block level cache.

>>>> 2. The majority of RapidDisk code focuses on the use of Volatile 
>>>> memory.
>>>> The support for Non-Volatile memory is a bit newer and there may be 
>>>> some
>>>> overlap here with the recently integrated pmem code. The only
>>>> advantage to
>>>> having this code within RapidDisk is to provide the user with the
>>>> ability
>>>> to manage both technologies simultaneously, through a single 
>>>> interface.
>>> Which really doesn't sound like a good enough reason to duplicate it.
>> I do not disagree with your comment here. This component does not have
>> to be patched into the mainline.
>>
>>>> 3. The RapidCache component is designed around the Non-Volatile
>>>> functionality of RapidDisk (hence the block-level Write-Through
>>>> caching).
>>>> It is also coded and optimized around the RapidDisk sizes/variables,
>>>> out-of-box. It is worth noting that I am in the process of expanding
>>>> this
>>>> module to add deduplication support. This will leverage RapidDisk's
>>>> ability
>>>> to allocate pages only when needed and reduce the cache's memory
>>>> footprint;
>>>> making more out of less.
>>> Still needs some code comparism to our existing two caching solutions.
>>>
>>> I'd love to see you go ahead with the dynamic ramdisk configuration as
>>> this is clearly a very useful feature.  A caching solution that is
>>> optimized for non-volatile memory does sound useful, but we'll still
>>> need a patch better explaining how it actually is as useful as it might
>>> sound.
>> CORRECTION: I meant to say Volatile and NOT Non-Volatile. RapidCache is
>> designed around Volatile memory. I guess I was a little to excited in my
>> response and I do apologize for that. I will provide a code comparison
>> in my next e-mail, after I go through the existing RAM drive code.
> To a certain extent, I see that as potentially less useful than 
> optimized for non-volatile memory.  While the current incarnation of 
> the pagecache in Linux could stand to have some serious performance 
> improvements (just think how fast things would be if we used ARC 
> instead of plain LRU), it does still do it's job well for most 
> workloads (although being able to tell the kernel to reserve some 
> portion of memory _just_ for the pagecache would be an interesting and 
> probably very useful feature).
>
My only concern with an ARC is CPU utilization. A lot more is required 
to manage two lists.