From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751594AbbI3O3r (ORCPT ); Wed, 30 Sep 2015 10:29:47 -0400 Received: from mout.perfora.net ([74.208.4.194]:53057 "EHLO mout.perfora.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751523AbbI3O3n (ORCPT ); Wed, 30 Sep 2015 10:29:43 -0400 Subject: Re: [PATCH] Patch to integrate RapidDisk and RapidCache RAM Drive / Caching modules into the kernel To: Austin S Hemmelgarn , Christoph Hellwig References: <1443374244.8013.7.camel@petros-ultrathin> <20150928064936.GA22280@infradead.org> <20150928162944.GA29562@infradead.org> <56096E90.2020000@petroskoutoupis.com> <560AA116.9030300@gmail.com> Cc: linux-kernel@vger.kernel.org, "devel@rapiddisk.org" From: Petros Koutoupis Message-ID: <560BF1DF.3000506@petroskoutoupis.com> Date: Wed, 30 Sep 2015 09:29:51 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <560AA116.9030300@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:Q/zIVbpQbARLsDsyyo+cOWiPKq39dbrzqfcHD7B0SQ8AEBfSnWH o+uvhviUXRx9xvXbqVwefaOQU4o1whOzk671uk1d1ItXLiEV73JRAbv6NPHmrCE84HZsByx Q/gGwmOJopZzdJCCSmaYeXitqDcoaQIVWbTg4/enITC22xBdV4EMVBibmpQFPPc1I0kl2BP xIBeamvjt5pBbvpkUPELQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:R4IeRAfb0ww=:ZIlHtUnyCd3qu+QXkZ3yt5 3Yi3kj6mqbekRiENOZlWgGG2qeMoVj+3OcKVWcYleuGb0xSkI1MDno0TafY1h+iMML/nUBR1a EDE2ccealei41brzP6+TZnTgHfbZrFjZtVtBtzROctXRXD6rQOR4GHaBHdZa44KaxaO6V9HuU iUK3g9n56vXsFL7vkUSoPw8X4r0xzV+CytQWN883YTb9wSDJH7eAx+6fZ4+hsE1oCKKrnpKOH PcACjMz4C4xb10xLIhRZbVkaH53O0IQDFuszLIol2u2BstFfQEMXlEV2XXS2WLBDMB7vUM39K ZTlKOwJ6gBE3Dv/Fz5exrVab15cjgJJYCe9R0oOWpuM21q/jH1zRmZovYcVUk2t55exsAVrX7 LWpKNrDuI/InFjUAxMrgSF/QUnLlrEo26i9TtnHGOYQ74scxGv/PcCZliRMh4X84RRx8lvf6f 2NFHLmLu+qr23M+LY3iGLBXUO1JrDP+AVVGriDgvMrFy5AHrPNcQvdFOLlORaF9EHabzi3De/ jdHPJp9mSd6AwbR1xmLq60gk4w3XXwUcH//WudDKJnYQO2vQXF2vfKV17pR8NJBBm9u9JL0zu pQ2cQKnDD2qmjK94v1CrJ0Qhs6lgQR9WtwnOjXl4psSk6G0txKczL+eBXTbo/vQUyYgAObt1s HdA2w1K+akf2UTlJrMP/FfKlQSgP97caNip1VaBYe+9OnusHRpV1rOamoIEJdMbVmEQgOUD1G djtSrrHMznStv1HE Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Christoph and Austin, You both have provided me with some valuable feedback. I will do what I can to clean this patch up and in turn apply the same dynamic functionality to the already in-kernel module. Also please see my replies below. On 9/29/15 9:32 AM, Austin S Hemmelgarn wrote: > On 2015-09-28 12:45, Petros Koutoupis wrote: >> Christoph, >> >> See my replies below.... >> >> On 9/28/15 11:29 AM, Christoph Hellwig wrote: >>> Hi Petros, >>> >>> On Mon, Sep 28, 2015 at 09:12:13AM -0500, Petros Koutoupis wrote: >>>> 1. Unlike the already mainline ramdisk driver, RapidDisk is designed >>>> to be >>>> managed dynamically. That is, instead of configuring a fixed number of >>>> volumes and volume sizes as compile/boot time variables, RapidDisk >>>> will >>>> allow you to add, remove, and resize your RAM drive(s) at runtime. >>>> Besides, >>>> the built in module is designed to work with smaller sizes in mind >>>> while >>>> RapidDisk focuses on larger sizes that can reach to the multiple >>>> Gigabytes >>>> or even Terabytes. Much like the built in module, it will allocate >>>> pages as >>>> they are needed which allows for over provisioning (not that it is >>>> advised) >>>> of volume sizes. >>> The ramdisk driver allows to selects sizes and count at module load >>> load. I agree that having runtime control would be even better, but >>> that's best done by adding a runtime interface to the existing driver >>> instead of duplicating it. >> I understand the concern and I will definitely scope out this approach, >> although at the moment, I am not sure how both approaches will play nice >> together. As mentioned above, the current implementation requires the >> predefined number of ram drives with the specified size to be configured >> at boot time (or compiled into the kernel). The only wiggle room I see >> for runtime control is resizing individual volumes. > Just because there is not code currently to do dynamic > allocation/freeing of ramdisks in the current driver doesn't mean that > it isn't possible, it just means that nobody has written code to do it > yet. This functionality would be extremely useful (I often use > ramdisks on a VM host as a small amount of very fast swap space for > the virtual machines). On top of that, the deduplication would be a > wonderful feature, although it may already be indirectly implemented > through KSM (that is, when KSM is on and configured to scan > everything, I'm not sure if it scans memory used by the ramdisks or not). > To my understanding KSM is only applied to KVM deployments. One way I have seen my caching module work is users/vendors have a block device, map it to a RapidDisk RAM drive as a RAM based Write-Through caching node and in turn export it via a traditional SAN. The idea behind adding deduplication to this module is to minimize the RAM drive footprint when used as a block level cache. >>>> 2. The majority of RapidDisk code focuses on the use of Volatile >>>> memory. >>>> The support for Non-Volatile memory is a bit newer and there may be >>>> some >>>> overlap here with the recently integrated pmem code. The only >>>> advantage to >>>> having this code within RapidDisk is to provide the user with the >>>> ability >>>> to manage both technologies simultaneously, through a single >>>> interface. >>> Which really doesn't sound like a good enough reason to duplicate it. >> I do not disagree with your comment here. This component does not have >> to be patched into the mainline. >> >>>> 3. The RapidCache component is designed around the Non-Volatile >>>> functionality of RapidDisk (hence the block-level Write-Through >>>> caching). >>>> It is also coded and optimized around the RapidDisk sizes/variables, >>>> out-of-box. It is worth noting that I am in the process of expanding >>>> this >>>> module to add deduplication support. This will leverage RapidDisk's >>>> ability >>>> to allocate pages only when needed and reduce the cache's memory >>>> footprint; >>>> making more out of less. >>> Still needs some code comparism to our existing two caching solutions. >>> >>> I'd love to see you go ahead with the dynamic ramdisk configuration as >>> this is clearly a very useful feature. A caching solution that is >>> optimized for non-volatile memory does sound useful, but we'll still >>> need a patch better explaining how it actually is as useful as it might >>> sound. >> CORRECTION: I meant to say Volatile and NOT Non-Volatile. RapidCache is >> designed around Volatile memory. I guess I was a little to excited in my >> response and I do apologize for that. I will provide a code comparison >> in my next e-mail, after I go through the existing RAM drive code. > To a certain extent, I see that as potentially less useful than > optimized for non-volatile memory. While the current incarnation of > the pagecache in Linux could stand to have some serious performance > improvements (just think how fast things would be if we used ARC > instead of plain LRU), it does still do it's job well for most > workloads (although being able to tell the kernel to reserve some > portion of memory _just_ for the pagecache would be an interesting and > probably very useful feature). > My only concern with an ARC is CPU utilization. A lot more is required to manage two lists.