From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935622AbcCRAbo (ORCPT ); Thu, 17 Mar 2016 20:31:44 -0400 Received: from LGEAMRELO11.lge.com ([156.147.23.51]:35564 "EHLO lgeamrelo11.lge.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933039AbcCRAbg (ORCPT ); Thu, 17 Mar 2016 20:31:36 -0400 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: minchan@kernel.org X-Original-SENDERIP: 165.244.98.204 X-Original-MAILFROM: minchan@kernel.org X-Original-SENDERIP: 10.177.223.161 X-Original-MAILFROM: minchan@kernel.org Date: Fri, 18 Mar 2016 09:32:36 +0900 From: Minchan Kim To: Sergey Senozhatsky CC: Sergey Senozhatsky , Andrew Morton , linux-kernel@vger.kernel.org Subject: Re: [PATCH] zram: export the number of available comp streams Message-ID: <20160318003236.GB2154@bbox> References: <1453809839-21705-1-git-send-email-sergey.senozhatsky@gmail.com> <20160129072842.GA30072@bbox> <20160201010157.GA1033@swordfish> MIME-Version: 1.0 In-Reply-To: <20160201010157.GA1033@swordfish> User-Agent: Mutt/1.5.21 (2010-09-15) X-MIMETrack: Itemize by SMTP Server on LGEKRMHUB06/LGE/LG Group(Release 8.5.3FP6|November 21, 2013) at 2016/03/18 09:31:33, Serialize by Router on LGEKRMHUB06/LGE/LG Group(Release 8.5.3FP6|November 21, 2013) at 2016/03/18 09:31:33, Serialize complete at 2016/03/18 09:31:33 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Sergey, I forgot this patch until now. Sorry about that. On Mon, Feb 01, 2016 at 10:02:48AM +0900, Sergey Senozhatsky wrote: > Hello Minchan, > > On (01/29/16 16:28), Minchan Kim wrote: > > Hello Sergey, > > > > Sorry to late response. Thesedays, I'm really busy with personal > > stuff. > > sure, no worries :) > > > On Tue, Jan 26, 2016 at 09:03:59PM +0900, Sergey Senozhatsky wrote: > > > I've been asked several very simple questions: > > > a) How can I ensure that zram uses (or used) several compression > > > streams? > > > > Why does he want to ensure several compression streams? > > As you know well, zram handle it dynamically. > > > > If zram cannot allocate more streams, it means the system is > > heavily fragmented or memory pressure at that time so there > > is no worth to add more stream, I think. > > > > Could you elaborate it more why he want to know it and what > > he expect from that? > > good questions. I believe mostly it's about fine-tuning on a > per-device basis, which is getting especially tricky when zram > devices are used as a sort of in-memory tmp storage for various > applications (black boxen). > > > > b) What is the current number of comp streams (how much memory > > > does zram *actually* use for compression streams, if there are > > > more than one stream)? > > > > Hmm, in the kernel, there are lots of example subsystem > > we cannot know exact memory usage. Why does the user want > > to know exact memory usage of zram? What is his concern? > > certainly true. probably some of those sub-systems/drivers have some > sort of LRU, or shrinker callbacks, to release unneeded memory back. > zram only allocates streams, and it basically hard to tell how many: > up to max_comp_streams, which can be larger than the number of cpus > on the system; because we keep preemption enabled (I didn't realize > that until I played with the patch) around > zcomp_strm_find()/zcomp_strm_release(): > > zram_bvec_write() > { > ... > zstrm = zcomp_strm_find(zram->comp); > >> can preempt > user_mem = kmap_atomic(page); > >> now atomic > zcomp_compress() > ... > kunmap_atomic() > >> can preempt > zcomp_strm_release() > ... > } > > so how many streams I can have on my old 4-cpus x86_64 box? > > 10? > yes. > > # cat /sys/block/zram0/mm_stat > 630484992 9288707 13103104 0 13103104 16240 0 10 > > 16? > yes. > > # cat /sys/block/zram0/mm_stat > 1893117952 25296718 31354880 0 31354880 15342 0 16 > > 21? > yes. > > # cat /sys/block/zram0/mm_stat > 1893167104 28499936 46616576 0 46616576 15330 0 21 > > do I need 21? may be no. do I nede 18? if 18 streams are needed only 10% > of the time (I can figure it out by doing repetitive cat zramX/mm_stat), > then I can set max_comp_streams to make 90% of applications happy, e.g. > max_comp_streams to 10, and save some memory. > Okay. Let's go back to zcomp design decade. As you remember, the reason we separated single and multi stream code was performance caused by locking scheme(ie, mutex_lock in single stream model was really fast than sleep/wakeup model in multi stream). If we could overcome that problem back then, we should have gone to multi stream code default. How about using *per-cpu* streams? I remember you wanted to create max number of comp streams statically although I didn't want at that time but I change my decision. Let's allocate comp stream statically but remove max_comp_streams knob. Instead, by default, zram alloctes number of streams according to the number of online CPU. So I think we can solve locking scheme issue in single stream , guarantee parallel level as well as enhancing performance with no locking. Downside with the approach is that unnecessary memory space reserve although zram might be used 1% of running system time. But we should give it up for other benefits(ie, simple code, removing max_comp_streams knob, no need to this your stat, guarantee parallel level, guarantee consumed memory space). What do you think about it?