From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F29C5C636CD for ; Mon, 30 Jan 2023 22:11:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 574826B0072; Mon, 30 Jan 2023 17:11:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5245A6B0073; Mon, 30 Jan 2023 17:11:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3EC2C6B0074; Mon, 30 Jan 2023 17:11:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 305C86B0072 for ; Mon, 30 Jan 2023 17:11:44 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F1E27C0BB7 for ; Mon, 30 Jan 2023 22:11:43 +0000 (UTC) X-FDA: 80412863286.19.C8B82BA Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) by imf07.hostedemail.com (Postfix) with ESMTP id 0349C4000F for ; Mon, 30 Jan 2023 22:11:41 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel-dk.20210112.gappssmtp.com header.s=20210112 header.b=nlmOc4Wi; spf=pass (imf07.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.47 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675116702; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QDmWTfugN+KPnRJSF8VFqZVl7659BFN6n85PK3By/Nc=; b=FjcQ2tX0+wWZnoNUtCCO38Nz425YeXBOnyb5cf7yLx4Ia78Kp23XWylqbqxCF6RacGOBdb sRLkv4oip4WFLt0ADj19hB+jgohfIdqFACykKVAAC7RZ/oZsv2oTACb2pW8mI3Isk7w9jZ Oj2WS8VJNelRFjoKEEFV6lU/GBJEfYc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel-dk.20210112.gappssmtp.com header.s=20210112 header.b=nlmOc4Wi; spf=pass (imf07.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.47 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675116702; a=rsa-sha256; cv=none; b=6c7DaF0e9V3BaabH4M06Hnz1TOOBd4FmyPzov5n9sniv7BaYBiqfM9GIcGHgWNjWqwNsNc /TSxTwV/EfYThT+2l9/W/veIC08kk7ez7375h/stxb22gRw77IlsL/0zM3eo9+n+qCsAsm bV/vbeq+R1kZgC8noaqWJU9og6JQOEk= Received: by mail-io1-f47.google.com with SMTP id r6so2530667ioj.5 for ; Mon, 30 Jan 2023 14:11:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=QDmWTfugN+KPnRJSF8VFqZVl7659BFN6n85PK3By/Nc=; b=nlmOc4WiFbAADi0988MPCHfj5QF4EnG9I8vaKRc2e6p1wgBDvQN8K85rsfHPq3bnzz +5Wpk2M14PHX4sLLXYbWPiNqjt5hNOjGnF0A0G7FcIUQhsUE7yp4WCS9ksaGv4X6PXuS TGw+Pnf5qwEw6KIrYBzOEok93sqvMiIvEBShjP1ZhXRg902LS2o4ZW1MHKIYzN43uKWR ETbp6zE6TY3BdA52DLjHc0etLXeq+J7Q1N22iMvAvtAOqF+xuRz7R9/ChivDlPnSg4mX p+bXJIJ59idbqVY+PP5h9kLZRWy8uR/o4OByjm8W/b8sF63cFmEMRB8lJRzLh4Hoh5SH 6+Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QDmWTfugN+KPnRJSF8VFqZVl7659BFN6n85PK3By/Nc=; b=kLuHoaP3mt7X+qsPndJVJwkYn7cpLxrkFm4iDM7RoXTCG6ZniBPvW3sn3QJ1k0lEU+ ux4p3pOviWx4N9cNUcaFkgHHLAumxVg+P5jWwzm3V+8wO1iQYzoCsIpE89fBWUi0Gg3I yXQkY7T5wPrFLGE6WHvoKePUblVulfvscqrHP7ryjsOgXfiAfdyReENBjZ6udMC6Lyj6 pyp/p+JzlAJSkNTWRotzBrtA3VrLxd/W9eVz50sGwfWsIYgNMrEoTReFIuo30mBIvjcP AvvBt168PJHZEMCZ49Pe52pq5oa+AqIsH4dIHMb7lyrBW98Zs9QohWdPHV6Wk11XXEzb HQPg== X-Gm-Message-State: AO0yUKU8mTf4zDa0eL1ixUdvBgKCE5bOJCNPj0o0wCXzAvCFXNq8k5qk Q91xt5Ee0Y3ugSki61NiBOrINQ== X-Google-Smtp-Source: AK7set9lGPWLFsiulveNFHus1v4u4pnnWzF0DBa0qziScPlEkynj8aHMSJd2IdwJIecjeXz0X86ddw== X-Received: by 2002:a05:6602:88f:b0:719:6a2:99d8 with SMTP id f15-20020a056602088f00b0071906a299d8mr1396318ioz.0.1675116701027; Mon, 30 Jan 2023 14:11:41 -0800 (PST) Received: from [192.168.1.94] ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id m4-20020a056638224400b003a958f51423sm5018787jas.167.2023.01.30.14.11.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 30 Jan 2023 14:11:40 -0800 (PST) Message-ID: <088e40fd-3fc7-77dd-a3de-0a2b097d3717@kernel.dk> Date: Mon, 30 Jan 2023 15:11:39 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux aarch64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: Re: [GIT PULL] iov_iter: Improve page extraction (pin or just list) Content-Language: en-US To: John Hubbard , David Howells Cc: Al Viro , Christoph Hellwig , Matthew Wilcox , Jan Kara , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Jeff Layton , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <3351099.1675077249@warthog.procyon.org.uk> From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 0349C4000F X-Stat-Signature: euhf89g5r98tw55kd6xbc16mk4dpadzi X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1675116701-354769 X-HE-Meta: U2FsdGVkX18ZrbaqoSShcw0TI8uEWPaACjGJScjhgi4cu+ts6bfHf4aYVF1knJlW3ZXWyvqH9W98KRd4/GOLfAyrSMiYiEwrYbMf/nVBBX4l3oE7huj60lgeNJtfhl6w4yYIis8YwnJG6151WXeNwy/yeSlzlJt2Ra8NcTKZ3ubqt+YeTpiwOiDuUssEJkutKIEedxFQ476cIBn//iB3gJqiqPmt+62flZMVmKlvDCVXj+OiJva4QV9GmzfdJkeN325T0zrCtDiMzZUv2Y0i4PtPHeAKa5BXhGgyrT3Q0IdHhtVrnTu7NOTYQkNE6H01ov5md9oPBtLVmskX5MFOO/p91rdYVNMpdkXPthIqzBwtb5LB/xUErucblMKNYUuMxBn9H6KK1amdvYwTX6d/LST4dX8ZxKFtnmlAdHEXI9zC3G15VmXpLCTEBrHE7OvBdW75EG459gDbrMAufvNxRFVrROIPeSNDBEfadzd8GPdAYXqEni+vLd/HLBK6ZxLGklzkq4A6Ms3xIBwsnnctD9c7ChqXe3NcRFJU7qYibf80hieG41Mk6K4kBVy+gn9fUFoZfJJzggkkx8/QMDIdulAGK456ZpScB5MCSwzUIHpQOJmVmO5i1YIXVoW5ze48OMh0CHhfuWUP1/vv7hn6Bb3XEGkj7pBr9l28TGEu2HZi/uTTUTicV0mrzveCeS16ABUVfWWfYpFXAaEprYbv9lve2iIhyOSLZZ5Qr8EFgmifkHqhXMxa55737iD9mtmdN6L4PitXzHnoBLvQiQcIaXLNCxoofdLzCTgzpGpFkP6lqLTikk+cChrWgSyOzcC71zkZm6foaQU1KYiMgRCAba+1g3RcsDJdP9p4SmyfJ+eFtVLGAfKlwElq79tufU2/GSGX7G8WvRqif3WaM+zKp+XvWCSKgGItKtnTlOR3dG2N6BJTMOAPBBSKjEVvn1CDIeGt5a2TJ9lScMEpVyO uN6se4Yc 3ofI4T+NiFdniqdmQdqv62ssFq9jIIcz4VAH7ce8Td50P3UVbcvMj5Sz+//XYoNdQr7YPQrlGJIMAipSUBA1iqeXTYUxeDc2333fj+27CrCNjGIQZj8o/FMY6t+tTgJvvvvMYpTA4oaP34ToVRtGYIo12uOEUrv7JGZXk9qf1QlY8kB2/af1gFp8eUSzFCgOgKEBVE+mMCzC22NRIErS+Z/LJIUNFvzXK2cu7EZ7/jFOAAxKtqsp7kIfKF2jxyfwHQO8SfNkKsbO3fFV2ULZzvqmxO6brLGQwS0Yu4jTAVt5jVMCp8D2CxGEHdaX2EzT+HBDkGHHgYTwyCxYfpiOMPg4oaSFQfFqEZeveVOlBrSOvsgZFnf1xyg2yflzbnWx3vZWr3PK6pI0rW4HJugkF2N4QFGC9uPqSH3f5hVlw27fjIj4vA54y3RlfWJoxjZPDYGN8qqnr1XVGS8Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/30/23 3:02?PM, John Hubbard wrote: > On 1/30/23 13:57, Jens Axboe wrote: >>> This does cause about a 2.7% regression for me, using O_DIRECT on a raw >>> block device. Looking at a perf diff, here's the top: >>> >>> +2.71% [kernel.vmlinux] [k] mod_node_page_state >>> +2.22% [kernel.vmlinux] [k] iov_iter_extract_pages >>> >>> and these two are gone: >>> >>> 2.14% [kernel.vmlinux] [k] __iov_iter_get_pages_alloc >>> 1.53% [kernel.vmlinux] [k] iov_iter_get_pages >>> >>> rest is mostly in the noise, but mod_node_page_state() sticks out like >>> a sore thumb. They seem to be caused by the node stat accounting done >>> in gup.c for FOLL_PIN. >> >> Confirmed just disabling the node_stat bits in mm/gup.c and now the >> performance is back to the same levels as before. >> >> An almost 3% regression is a bit hard to swallow... > > This is something that we say when adding pin_user_pages_fast(), > yes. I doubt that I can quickly find the email thread, but we > measured it and weren't immediately able to come up with a way > to make it faster. > > At this point, it's a good time to consider if there is any > way to speed it up. But I wanted to confirm that you're absolutely > right: the measurement sounds about right, and that's also the > hotspot that we say, too. >From spending all of 5 minutes on this, it must be due to exceeding the pcp stat_threashold, as we then end up doing two atomic_long_adds(). Looking at proc, looks like it's 108. And with this test, then we're hitting that slow path ~80k/second. Uhm... -- Jens Axboe