From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.7 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,FSL_HELO_FAKE,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A242C433E0 for ; Mon, 21 Dec 2020 20:22:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A2E78224D4 for ; Mon, 21 Dec 2020 20:22:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2E78224D4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 116D06B005C; Mon, 21 Dec 2020 15:22:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0ED076B0068; Mon, 21 Dec 2020 15:22:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0384B6B006C; Mon, 21 Dec 2020 15:22:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0206.hostedemail.com [216.40.44.206]) by kanga.kvack.org (Postfix) with ESMTP id DB6316B005C for ; Mon, 21 Dec 2020 15:22:12 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A8C841EFD for ; Mon, 21 Dec 2020 20:22:12 +0000 (UTC) X-FDA: 77618411304.25.debt31_4606aca2745a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 731E01804E532 for ; Mon, 21 Dec 2020 20:22:12 +0000 (UTC) X-HE-Tag: debt31_4606aca2745a X-Filterd-Recvd-Size: 7487 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Mon, 21 Dec 2020 20:22:11 +0000 (UTC) Received: by mail-pf1-f169.google.com with SMTP id x126so7081960pfc.7 for ; Mon, 21 Dec 2020 12:22:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=2i5d/rF9ghqWdmLNxdkBGDo6QlVvN3s6AnXqOUeYid8=; b=cVaoECoIkuXobTWAf62XCYvhfwS4rQv+P+YxwX/xR33bAJOSj2faSEGhR4uq3JWGkh DyrrUGAxbF9oS7bFlko8TXBj5irWAmrnU2w8HbqDwzrS00Xn4nN4tgoOZuRPZ7rMkppJ MvHSnIj6J8wV/JdSMbCxqK7zGp9CBGb8JXuJXwVW4QoGCpZ6QDMyps2wp6DO9AI1lYYF W9fSCvGFaC2HVphlJsrCLmJ5/qrGl1V3EESvKklVvTlU1ymEgIfhMi0dtdakLLdy6NAf vzWvGDyj77TlYHv7TVANdT7wdzIHf8ChzzFIROC8wei22BEpS3Q11f40hsg9Bj1zPt+g ek8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=2i5d/rF9ghqWdmLNxdkBGDo6QlVvN3s6AnXqOUeYid8=; b=bkxOyZOd2BfJLKe5hT5t0i+c3OcDhfngtd0UYjjJFNkZrn33/9/NcChUyAtruhKwJd 99Hbyf1liYLDn3bbKA9sYu1krO+PQBVPsWfTQKFjl5epExN4aUeUhziWQQguxXiifL6k UjT92vRmyz+uVKYBRTvU14zYpS4Q/+f3wCB41KtRSUOKb17CgcHhR1NgPiaVTgXS5zjR UaiGVzWDeZnMYqmUf+wG9Qrtg7ahv1INsaWbFdIg6uglr+Efiv5rFzPRbDpxjmH6DLab t2gVoecZrLN4A7ddbMwOSPp9M0wcNqd4+LBcm7B9LJpC/5BqaUmJ01CYaBXMeCoJeFv4 0xaQ== X-Gm-Message-State: AOAM531P7dEVQoxwWeP2JYzl6K337FMBLCgJmhaIxrdBtwEUDnwR53rU VrZ+AUWOkNz62oqfie6ORBk= X-Google-Smtp-Source: ABdhPJxCLY8So8MauqJ1pVZJ4ctDHxHp7f8JrxOKjZemM1g+2+d4lA2vlISI2tFMPDpEbIl7Hqu11g== X-Received: by 2002:a05:6a00:796:b029:1ad:6394:432d with SMTP id g22-20020a056a000796b02901ad6394432dmr8828748pfu.36.1608582130907; Mon, 21 Dec 2020 12:22:10 -0800 (PST) Received: from google.com ([2620:15c:211:201:7220:84ff:fe09:5e58]) by smtp.gmail.com with ESMTPSA id w2sm17621933pfj.110.2020.12.21.12.22.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Dec 2020 12:22:09 -0800 (PST) Date: Mon, 21 Dec 2020 12:22:07 -0800 From: Minchan Kim To: Vitaly Wool Cc: Mike Galbraith , LKML , linux-mm , Barry Song , Sebastian Andrzej Siewior , NitinGupta , sergey.senozhatsky.work@gmail.com, Andrew Morton , shakeelb@google.com Subject: Re: [PATCH] zsmalloc: do not use bit_spin_lock Message-ID: References: <18669bd607ae9efbf4e00e36532c7aa167d0fa12.camel@gmx.de> <20201220002228.38697-1-vitaly.wool@konsulko.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Dec 21, 2020 at 08:20:26PM +0100, Vitaly Wool wrote: > On Mon, Dec 21, 2020 at 6:24 PM Minchan Kim wrote: > > > > On Sun, Dec 20, 2020 at 02:22:28AM +0200, Vitaly Wool wrote: > > > zsmalloc takes bit spinlock in its _map() callback and releases it > > > only in unmap() which is unsafe and leads to zswap complaining > > > about scheduling in atomic context. > > > > > > To fix that and to improve RT properties of zsmalloc, remove that > > > bit spinlock completely and use a bit flag instead. > > > > I don't want to use such open code for the lock. > > > > I see from Mike's patch, recent zswap change introduced the lockdep > > splat bug and you want to improve zsmalloc to fix the zswap bug and > > introduce this patch with allowing preemption enabling. > > This understanding is upside down. The code in zswap you are referring > to is not buggy. You may claim that it is suboptimal but there is > nothing wrong in taking a mutex. No, it's surely break from zswap since zpool/zsmalloc has worked like this and now you are saying "nothing wrong" even though it breaks the rule. > > > https://lore.kernel.org/linux-mm/fae85e4440a8ef6f13192476bd33a4826416fc58.camel@gmx.de/ > > > > zs_[un/map]_object is designed to be used in fast path(i.e., > > zs_map_object/4K page copy/zs_unmap_object) so the spinlock is > > perfectly fine for API point of view. However, zswap introduced > > using the API with mutex_lock/crypto_wait_req where allowing > > preemption, which was wrong. > > Taking a spinlock in one callback and releasing it in another is > unsafe and error prone. What if unmap was called on completion of a > DMA-like transfer from another context, like a threaded IRQ handler? > In that case this spinlock might never be released. > > Anyway I can come up with a zswap patch explicitly stating that > zsmalloc is not fully compliant with zswap / zpool API to avoid > confusion for the time being. Would that be ok with you? It's your call since you are maintainer of zswap now and you are breaking the rule we have kept for a long time. > > Best regards, > Vitaly > > > Furthermore, the zs_map_object already has a few more places where > > disablepreemptions(migrate_read_lock, get_cpu_var and kmap_atomic). > > > > Without making those locks preemptible all at once, zswap will still > > see the lockdep warning. > > > > > > > > Signed-off-by: Vitaly Wool > > > --- > > > mm/zsmalloc.c | 13 ++++++++----- > > > 1 file changed, 8 insertions(+), 5 deletions(-) > > > > > > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c > > > index 7289f502ffac..ff26546a7fed 100644 > > > --- a/mm/zsmalloc.c > > > +++ b/mm/zsmalloc.c > > > @@ -876,22 +876,25 @@ static unsigned long obj_to_head(struct page *page, void *obj) > > > > > > static inline int testpin_tag(unsigned long handle) > > > { > > > - return bit_spin_is_locked(HANDLE_PIN_BIT, (unsigned long *)handle); > > > + return test_bit(HANDLE_PIN_BIT, (unsigned long *)handle); > > > } > > > > > > static inline int trypin_tag(unsigned long handle) > > > { > > > - return bit_spin_trylock(HANDLE_PIN_BIT, (unsigned long *)handle); > > > + return !test_and_set_bit(HANDLE_PIN_BIT, (unsigned long *)handle); > > > } > > > > > > -static void pin_tag(unsigned long handle) __acquires(bitlock) > > > +static void pin_tag(unsigned long handle) > > > { > > > - bit_spin_lock(HANDLE_PIN_BIT, (unsigned long *)handle); > > > + preempt_disable(); > > > + while(test_and_set_bit(HANDLE_PIN_BIT, (unsigned long *)handle)) > > > + cpu_relax(); > > > + preempt_enable(); > > > } > > > > > > static void unpin_tag(unsigned long handle) __releases(bitlock) > > > { > > > - bit_spin_unlock(HANDLE_PIN_BIT, (unsigned long *)handle); > > > + clear_bit(HANDLE_PIN_BIT, (unsigned long *)handle); > > > } > > > > > > static void reset_page(struct page *page) > > > -- > > > 2.20.1 > > >