From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F28A0C433EF for ; Sat, 12 Feb 2022 21:02:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vl5NdwbjiVi7UklutKAe/DQ3manN8U1t65+q3itE8pM=; b=aSXVCger3AAqkH ZC9dykzPRBzJ91jsxspk3r9+1YZNYm0t25ErEXjKPbrdX9aOYg80nZ/u9qAT3eW3qXhL8wtDCLBNF L30TOXB9kWreYhh3/a8aXU+g/+/UzsqH7uoELrzcw0qoNfE/nSOrpO26SfQUeDamp7mBjJlmFRPYg GwfkZWJPyNBj/S9zQ1/MKW4nxzg9ocvB2XXtyyluUJNeYwNJAhblTVebsnV1wmo9yqjQLJBlNTJkB Yug/0gbX423ItjQVp0neRdpBoPuz5PG9kE6IcxjRszmLODAeROwCvpJjHw3VbpJuxeS43VdSIiF0R Ai9OFF7u6cz3k70tvrZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nIzWS-00AY6z-8r; Sat, 12 Feb 2022 21:01:36 +0000 Received: from mail-io1-xd2e.google.com ([2607:f8b0:4864:20::d2e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nIzWO-00AY4I-K1 for linux-arm-kernel@lists.infradead.org; Sat, 12 Feb 2022 21:01:34 +0000 Received: by mail-io1-xd2e.google.com with SMTP id c188so15572694iof.6 for ; Sat, 12 Feb 2022 13:01:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=pMbNysZicDjla5m+LqKnIivTQyASvMT8jxUwCbQWv1g=; b=riVGorTeXgSv6f6PSqT26L43SORlysK2m4uRC/QszDsHk26fohbl3AcXeAU1z5WReD NiLXebj+jUjwxHFaizCF5lM+/IPGT9AeDHCQP7Ki4Nq39KGTQ668UcwNvKBImkPaoHXk KmbjeDZw7fppxgwkgRuh9xN/64WYlgfkC6a5u1qi/mrP2cuMCkW/R7DJwRORH6OWAt7P ethnE+skfclWxvZ36JWfsq6I+25qukq2eaM6QuAD0d/SybO6tkaG8PsRMLAk7yUPMg6G WyCg8Q53Rqg73T6F0bVKystdOEWJIJIs3XjVCZVbtZJJIgeFp3D7X2O9KlWWJvlNgBvj KumA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=pMbNysZicDjla5m+LqKnIivTQyASvMT8jxUwCbQWv1g=; b=5Ob1im8srDY1Yb1YAIqQO+HrtY0lx4s1z8k7T7IYCgPyptaIXvj92LGx8lBlgdTdWA +w2OsuTHLmfUKvAJA8UDxRAXDfymJ8i7OmAprOSqysSqD0Q9AU7sQkNtYDabsXbo3Ktl +Hmy42AT1Bpq80F8H6jq/jkzkws0jALETnmxEiaF5xJ/NZf/qSb+hClbZNqdhyE1p9U7 PNlEIZ86JZ0Qzy+qDNbRC6UppAElzBSKOsSFSbwhoGA6U/4mad/yvpqKPTjuEcxY0glC iAEjgKC1umMNFqVeSZaLURDeT32dszjYrfVlRYzFZ50zwiHSUsm0hQnZ98yLAiIffzVb xTHw== X-Gm-Message-State: AOAM532o4Db0v5NMdrFPCpXdpe23K1DOlRYcjRNLYsxZ69jko37gSO5W ouPVw5NHFHrSI4Ma8qz5Dlad3g== X-Google-Smtp-Source: ABdhPJzPD41+d7IoHgG5Q9TFesNC7mfL4rvQpeFd/BtV8ZyYVWurBH52if7QSOBzPeihv02iT4Y7qQ== X-Received: by 2002:a05:6638:22c6:: with SMTP id j6mr3979977jat.216.1644699688189; Sat, 12 Feb 2022 13:01:28 -0800 (PST) Received: from google.com ([2620:15c:183:200:7f5c:29f4:d1c2:e1f6]) by smtp.gmail.com with ESMTPSA id v8sm1703481iox.53.2022.02.12.13.01.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 12 Feb 2022 13:01:27 -0800 (PST) Date: Sat, 12 Feb 2022 14:01:23 -0700 From: Yu Zhao To: Alexey Avramov Cc: 21cnbao@gmail.com, Michael@michaellarabel.com, ak@linux.intel.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, axboe@kernel.dk, catalin.marinas@arm.com, corbet@lwn.net, dave.hansen@linux.intel.com, hannes@cmpxchg.org, hdanton@sina.com, jsbarnes@google.com, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mgorman@suse.de, mhocko@kernel.org, page-reclaim@google.com, riel@surriel.com, rppt@kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, will@kernel.org, willy@infradead.org, x86@kernel.org, ying.huang@intel.com Subject: Re: [PATCH v7 00/12] Multigenerational LRU Framework Message-ID: References: <20220208081902.3550911-1-yuzhao@google.com> <20220212051219.183d1baf@PC> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220212051219.183d1baf@PC> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220212_130132_668831_4451DD74 X-CRM114-Status: GOOD ( 13.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sat, Feb 12, 2022 at 05:12:19AM +0900, Alexey Avramov wrote: > Aggressive swapping even with vm.swappiness=1 with MGLRU > ======================================================== > > Reading a large mmapped file leads to a super agressive swapping. > Reducing vm.swappiness even to 1 does not have effect. Mind explaining why you think it's "super agressive"? I assume you expected a different behavior that would perform better. If so, please spell it out. > Demo: https://www.youtube.com/watch?v=J81kwJeuW58 > > Linux 5.17-rc3, Multigenerational LRU v7, > vm.swappiness=1, MemTotal: 11.5 GiB. > > $ cache-bench -r 35000 -m1 -b1 -p1 -f test20000 > Reading mmapped file (file size: 20000 MiB) > cache-bench v0.2.0: https://github.com/hakavlad/cache-bench Writing your own benchmark is a good exercise but fio is the standard benchmark in this case. Please use it with --ioengine=mmap. > Swapping started with MemAvailable=71%. > At the end 33 GiB was swapped out when MemAvailable=60%. > > Is it OK? MemAvailable is an estimate (free + page cache), and it doesn't imply any reclaim preferences. In the worst case scenario, e.g., out of swap space, MemAvailable *may* be reclaimed. Here is my benchmark result with file mmap + *high* swap usage. Ram disk was used to reduce the variance in the result (and SSD wear out if you care). More details on additional configurations here: https://lore.kernel.org/linux-mm/20220208081902.3550911-6-yuzhao@google.com/ Mixed workloads: fio (buffered I/O): +13% IOPS BW 5.17-rc3: 275k 1075MiB/s v7: 313k 1222MiB/s memcached (anon): +12% Ops/sec KB/sec 5.17-rc3: 511282.72 19861.04 v7: 572408.80 22235.49 cat mmap.sh systemctl restart memcached swapoff -a umount /mnt rmmod brd modprobe brd rd_nr=2 rd_size=56623104 mkswap /dev/ram0 swapon /dev/ram0 mkfs.ext4 /dev/ram1 mount -t ext4 /dev/ram1 /mnt memtier_benchmark -S /var/run/memcached/memcached.sock \ -P memcache_binary -n allkeys --key-minimum=1 \ --key-maximum=50000000 --key-pattern=P:P -c 1 \ -t 36 --ratio 1:0 --pipeline 8 -d 2000 sysctl vm.overcommit_memory=1 fio -name=mglru --numjobs=36 --directory=/mnt --size=1408m \ --buffered=1 --ioengine=mmap --iodepth=128 --iodepth_batch_submit=32 \ --iodepth_batch_complete=32 --rw=randread --random_distribution=random \ --norandommap --time_based --ramp_time=10m --runtime=990m \ --group_reporting & pid=$! sleep 200 memcached.sock -P memcache_binary -n allkeys --key-minimum=1 \ --key-maximum=50000000 --key-pattern=R:R -c 1 -t 36 --ratio 0:1 \ --pipeline 8 --randomize --distinct-client-seed kill -INT $pid wait _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel