From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 478D7C2D0C6 for ; Thu, 12 Dec 2019 01:29:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1BCE322B48 for ; Thu, 12 Dec 2019 01:29:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="tBcfwgxJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727473AbfLLB3w (ORCPT ); Wed, 11 Dec 2019 20:29:52 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:38987 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727391AbfLLB3v (ORCPT ); Wed, 11 Dec 2019 20:29:51 -0500 Received: by mail-pg1-f193.google.com with SMTP id b137so260958pga.6 for ; Wed, 11 Dec 2019 17:29:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=DhyX9yKmk3Bjq16+cH6CFIH0oBue170bHYeXiMOQ8dA=; b=tBcfwgxJPBhRRcAGhCQgQbyfTfpQsyiGY9DGe/XHH2/qbpQARxKq7yWjGdvqo1IllS dEWH4g1714qg29h4GEZigEfbqrKvxMzRE9LNVpEFK2NyrdjbNtkxs6UWbtUszECnOQ0D VEoKTKtnfhk/vEZGeDpWMdwNOGh8La+bbktv7kvota82pgoXLzk3sL28vntZlDhqgkLX +EyfDp8OYP1xKcWrZTib+Ng+VSUqbKeEhhTBmOwpw+Ojk7X5nKqyrRMXwloZOmKkzPiZ +9yGgbU006bu6ndm0cPeWABAqJMpqOiOI+vxaISk+h/Wium9aaoWX5gz51q7TBIt5Pl3 er3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=DhyX9yKmk3Bjq16+cH6CFIH0oBue170bHYeXiMOQ8dA=; b=J4IMAnGBfAiTrtQ96Nqb/TQ3eqEzHt536PMngLsY7yg4WLi5gsyHy8bFe1y+5zABnG +/N6IlIIHSo5jHkjCMePsz2cPR1CZhI8o+WvqZYtNMTOMh8Epo7/41ym448S5O40Abif 2HiLy2HNR11ewwQp00KBF0mzSVMev/9sDoRSYpQ8i0F2N1pUUeKAj/Xk/rcLJJdHi1nP /81CMn919Q9qUFSzF+FjmPqgOt+LV1Zf6OFlV83cEeECgDHjv1+0Bd67wMV+E7zCo0mV GjjcfKmveK3yL45xb3g2YNlcg5XD3uyQq8bxhMUS92PLZJRYMGXLj5ngjyVAzhGG9SS2 TaEg== X-Gm-Message-State: APjAAAXSnUDqWEpQSsIN9JWfxNhi+aj9KONfePCI16CBFnYcs5Edxaof BsxKGuT9QykFfOaVjPP08HszYQ== X-Google-Smtp-Source: APXvYqwIZT80qb6KK6M7DWfi9Y0s3aK7ho5OWU9hmanRrqTeBJgLbXHTk13enGqUZ5WC+TJcO9HU0Q== X-Received: by 2002:a63:fd0a:: with SMTP id d10mr7322984pgh.197.1576114191223; Wed, 11 Dec 2019 17:29:51 -0800 (PST) Received: from [192.168.1.188] ([66.219.217.145]) by smtp.gmail.com with ESMTPSA id z14sm2580858pfg.57.2019.12.11.17.29.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Dec 2019 17:29:50 -0800 (PST) Subject: Re: [PATCHSET v3 0/5] Support for RWF_UNCACHED To: Linus Torvalds Cc: Linux-MM , linux-fsdevel , linux-block , Matthew Wilcox , Chris Mason , Dave Chinner , Johannes Weiner References: <20191211152943.2933-1-axboe@kernel.dk> <0d4e3954-c467-30a7-5a8e-7c4180275533@kernel.dk> <1c93194a-ed91-c3aa-deb5-a3394805defb@kernel.dk> From: Jens Axboe Message-ID: Date: Wed, 11 Dec 2019 18:29:46 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On 12/11/19 6:22 PM, Linus Torvalds wrote: > On Wed, Dec 11, 2019 at 5:11 PM Jens Axboe wrote: >> >> 15K is likely too slow to really show an issue, I'm afraid. The 970 >> is no slouch, but your crypt setup will likely hamper it a lot. You >> don't have a non-encrypted partition on it? > > No. I normally don't need all that much disk, so I've never upgraded > my ssd from the 512G size. > > Which means that it's actually half full or so, and I never felt like > "I should keep an unencrypted partition for IO testing", since I don't > generally _do_ any IO testing. > > I can get my load up with "numjobs=8" and get my iops up to the 100k > range, though. > > But kswapd doesn't much seem to care, the CPU percentage actually does > _down_ to 0.39% when I try that. Probably simply because now my CPU's > are busy, so they are running at 4.7Ghz instead of the 800Mhz "mostly > idle" state ... > > I guess I should be happy. It does mean that the situation you see > isn't exactly the normal case. I understand why you want to do the > non-cached case, but the case I think it the worrisome one is the > regular buffered one, so that's what I'm testing (not even trying the > noaccess patches). > > So from your report I went "uhhuh, that sounds like a bug". And it > appears that it largely isn't - you're seeing it because of pushing > the IO subsystem by another order of magnitude (and then I agree that > "under those kinds of IO loads, caching just won't help") I'd very much argue that it IS a bug, maybe just doesn't show on your system. My test box is a pretty standard 2 socket system, 24 cores / 48 threads, 2 nodes. The last numbers I sent were 100K IOPS, so nothing crazy, and granted that's only 10% kswapd cpu time, but that still seems very high for those kinds of rates. I'm surprised you see essentially no kswapd time for the same data rate. We'll keep poking here, I know Johannes is spending some time looking into the reclaim side. -- Jens Axboe