From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57A21C433FE for ; Thu, 3 Nov 2022 20:22:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C3396B0072; Thu, 3 Nov 2022 16:22:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9738F6B0073; Thu, 3 Nov 2022 16:22:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83B016B0074; Thu, 3 Nov 2022 16:22:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 713EF6B0072 for ; Thu, 3 Nov 2022 16:22:46 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3D00B16014B for ; Thu, 3 Nov 2022 20:22:46 +0000 (UTC) X-FDA: 80093254332.21.1B98F45 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf10.hostedemail.com (Postfix) with ESMTP id E2A7DC0004 for ; Thu, 3 Nov 2022 20:22:45 +0000 (UTC) Received: by mail-pl1-f176.google.com with SMTP id g24so3017626plq.3 for ; Thu, 03 Nov 2022 13:22:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=FEPi06UgKprQnUM8Zfxe+0RaYyxeoCxUMy8+FyU8lNQ=; b=RLKl3RRCVOU7T+04We+GPkeg1MV7TVfa1A5pCUTe62eVzkeqSRZ14bbUPPKHGpvEHN fFKDMKK5lJzDbyDVuSaCkp2S5SNelBWvc13FcdFUSdpQkRkpQes5UkdxR5gTd/RMfez6 8EH06nkH29bLKXDDUh6L7YFItGjAO0Etoy/9DYZ/YMWuTfds96NZqgthvdWPm0b5dhsL AMddYYfh0ETFAIThd43v0wF1vplvY9D8iC1J9hF8YB8JVOcju4WnfMg+dgsb9dPqVaib JME03sotYrLf7jtqgHSVxqbKCabvTUj1uHH0B+rn4Mbwf0r6/NpUIzTg/gvQ1AqR0J2t tC6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FEPi06UgKprQnUM8Zfxe+0RaYyxeoCxUMy8+FyU8lNQ=; b=CrIMlJj70aMnrOhS1w8ESwh10+DBiOtFamTruECwW8sdklgwAWo4J0QnpOG5mb4+lk 9QMzQmfTGAuxRGckVDq6k8hBvtGcmxmonMJxWsw9pbkdsvIeeJ2dn0ls2PIAovgRcnSH wgM4aHjj/78YVMwX3beFBqShv3N2vO7A4/KzoRvBh5rbP+Wp8UA7JWdcWGtmQAempC4U 8cOJvSOD2wO7OEYNvjbd0GeEHVr5STkLIB0jZ+u4HZ3PFrE5covlwNR3zOnCa2stSRVE 9ApMx9vzC/NaX5AjH9Co2BcGQS++PSytmzlTO1i2c0XCTmy4NPcbTkMpPhAsH0gbP2tx 99rw== X-Gm-Message-State: ACrzQf0P+N48X6QmUmkUu9jGpBfqfAvWySbXBdy5tvh/fc+ZMbQelCOh uKoXBtA3/62EG9CyaFd2yGw= X-Google-Smtp-Source: AMsMyM6WUL6vr+Xs7PtCW0Yz1RJilmg91uXuCh7hjHHqZ/JRI/3uDwJQA1gl5rqZdFuDXmNMyfpD+A== X-Received: by 2002:a17:902:ec01:b0:186:748f:e8b6 with SMTP id l1-20020a170902ec0100b00186748fe8b6mr31485122pld.131.1667506964704; Thu, 03 Nov 2022 13:22:44 -0700 (PDT) Received: from google.com ([2620:15c:211:201:3d65:7dc2:c62a:5d98]) by smtp.gmail.com with ESMTPSA id 123-20020a621781000000b00562784609fbsm1111632pfx.209.2022.11.03.13.22.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Nov 2022 13:22:44 -0700 (PDT) Date: Thu, 3 Nov 2022 13:22:42 -0700 From: Minchan Kim To: Johannes Weiner Cc: Sergey Senozhatsky , Nhat Pham , akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, ngupta@vflare.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: Re: [PATCH 2/5] zsmalloc: Consolidate zs_pool's migrate_lock and size_class's locks Message-ID: References: <20221026200613.1031261-1-nphamcs@gmail.com> <20221026200613.1031261-3-nphamcs@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667506965; a=rsa-sha256; cv=none; b=aIyh6TZI7VYuasVP50YgJty7aWBYuDHgV2pPRUGaJNbln5nwjupE3mJZKjVXeD9BHK6M06 FxyTa9SmLKfpJn0aFQL1zfh9GVnBx3j54A1ouX9hNthn1ZAz9oWNKItE0D4JDKRd9ux/Da ZQQcDS8ErzFuxjUIAs5zbuFPJ1UuoFo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=RLKl3RRC; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=pass (imf10.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667506965; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FEPi06UgKprQnUM8Zfxe+0RaYyxeoCxUMy8+FyU8lNQ=; b=ChhHzvZ9rWKQG8jsXopBDQPOYwBwpM1dsfAw4PkMIwUI6kUy3rIAbHi1t/9SqNskRVy5Mx CtVvezWFje+bgMRD+mSV7NXLzCGJlrwhMHtCD7qLnKH4rYT0SCwmqjyh+RgsSmK5p1r33t +ynl/vqsgoIwx79Nlg4PRn10R3wuk6c= X-Rspamd-Server: rspam02 X-Rspam-User: Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=RLKl3RRC; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=pass (imf10.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com X-Stat-Signature: ibnzkhwamnckb675pr91ma5b5ycixmsi X-Rspamd-Queue-Id: E2A7DC0004 X-HE-Tag: 1667506965-568356 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Nov 03, 2022 at 02:08:01PM -0400, Johannes Weiner wrote: < snip > > > > > I am also worry about that LRU stuff should be part of allocator > > > > instead of higher level. > > > > > > I'm sorry, but that's not a reasonable objection. > > > > > > These patches implement a core feature of being a zswap backend, using > > > standard LRU and locking techniques established by the other backends. > > > > > > I don't disagree that it would nicer if zswap had a strong abstraction > > > for backend pages and a generalized LRU. But that is major surgery on > > > a codebase of over 6,500 lines. It's not a reasonable ask to change > > > all that first before implementing a basic feature that's useful now. > > > > With same logic, folks added the LRU logic into their allocators > > without the effort considering moving the LRU into upper layer. > > > > And then trend is still going on since I have seen multiple times > > people are trying to add more allocators. So if it's not a reasonable > > ask to consier, we couldn't stop the trend in the end. > > So there is actually an ongoing effort to do that. Yosry and I have > spent quite some time on coming up with an LRU design that's > independent from compression policy over email and at Plumbers. I am really glad to hear somebody is working toward right direction. > > My concern is more about the order of doing things: > > 1. The missing writeback support is a gaping hole in zsmalloc, which > affects production systems. A generalized LRU list is a good idea, > but it's a huge task that from a user pov really is not > critical. Even from a kernel dev / maintainer POV, there are bigger > fish to fry in the zswap code base and the backends than this. Even though I believe the general LRU in the swap subsystem is way to go, I was about to suggesting putting the LRU logic in the zswap layer to stop this trend since it's not too difficult at my first glance(Sure, I might miss something clear there). However, if you guys are working toward the generalized direction, I am totally in favor of the approach and looking forward to seeing the project under expectation that we will clean up all the duplicated logic, fixing the weird layering and then finally supports hierarchical swap writeback for any combinations of swap devices. > > 2. Refactoring existing functionality is much easier than writing > generalized code that simultaneously enables new behavior. zsmalloc > is the most complex of our backends. To make its LRU writeback work > we had to patch zswap's ->map ordering to accomodate it, e.g. Such > tricky changes are easier to make and test incrementally. > > The generalized LRU project will hugely benefit from already having > a proven writeback implementation in zsmalloc, because then all the > requirements in zswap and zsmalloc will be in black and white. Agreed if we are working toward the right direction and this work could help to fill the gap of the hole until we can see all the requirements and achieve it. > > > > I get that your main interest is zram, and so this feature isn't of > > > interest to you. But zram isn't the only user, nor is it the primary > > > > I am interest to the feature but my interest is more of general swap > > layer to manage the LRU so that it could support any hierarchy among > > swap devices, not only zswap. > > I think we're on the same page about the longer term goals. Fingers crossed.