From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3547C433F5 for ; Mon, 15 Nov 2021 18:59:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 829A36368C for ; Mon, 15 Nov 2021 18:59:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 829A36368C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 2DC9A6B007D; Mon, 15 Nov 2021 13:59:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2176E6B007E; Mon, 15 Nov 2021 13:59:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DEF86B0080; Mon, 15 Nov 2021 13:59:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id F3A876B007D for ; Mon, 15 Nov 2021 13:59:18 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8AE03181AEF21 for ; Mon, 15 Nov 2021 18:59:18 +0000 (UTC) X-FDA: 78812077596.29.BDEE2C3 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf22.hostedemail.com (Postfix) with ESMTP id 35F901923 for ; Mon, 15 Nov 2021 18:59:18 +0000 (UTC) Received: by mail-pf1-f177.google.com with SMTP id r130so15887454pfc.1 for ; Mon, 15 Nov 2021 10:59:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pSUi2+kpqxZMVeYqD80ROQDgqt7JZMMjZIIlCp0yEtk=; b=G6MoOgbvzNNQtqZiSD8/K3soql42kJiSY7PB/Q5GykBx4HKqjTY0OqiP+f4IsINuOk XOUcTx26GxXg5dKW4gjtYDSezec+I4Q1N6xbJasQwfwCqJ+VqnT+09yGNTSOnNT4Kpmc NAIAIuszweVULkidU0CW1sqGbgeuJB1eZJWaRoKZ2/Osi5J1pAy/Mgu438UoSjeRxFD2 7t6dNrUXBw6Cx+ET9k4gc+MbkM/GprmPCED/gcVSB/e+Kjv5EChStMjCqrRuNX7bcSaw iTbwAqKg6+vs8Cgya0PjPCefhoRPuvbjrEbV1gH5PI7QZtN3vuMtfawynRKCIBQSh27u wpNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=pSUi2+kpqxZMVeYqD80ROQDgqt7JZMMjZIIlCp0yEtk=; b=DUvq8LZJfaWh2Dyp9ccWBPyyU+Kj2HzbEaLe4kBHQOoFnyhurf6PAOzK9HAeZZK326 tw5cykBUw5lcTA21YJ4EphTTwaCuwOdRfTamWR96tKO2mWLh0fbovaFIjVfXsbGy2/+g WW0M2Ig/UDmi7dEuRq1SwcXWcNveaMx7xZHnbHyIAMLIcDVNOfuz3EolxFXNA/9pEL+c rZOg7V5QLbvGcRsGIyswpj9zzPoRoJhMVwsOejpHjEP2geHwMyfJIRd/eQ5y9cLb298A 7HBZi6jSS0l2K+SVhSqRp+mbsqbAtU6DOAQpKI2EXFoLcwwTad2ucQH6CoiXE2mgLQ5d zzhQ== X-Gm-Message-State: AOAM532jRKCI9MsBbCs1mZRGsUjAF2gLsvTZN7urPMrFVBXgLAuKp08p bGZBmChJg1ehKeydnnvrUIM= X-Google-Smtp-Source: ABdhPJyTVlrU2ufUgiTIqfkdmqbn+i2qnwloef8v3S29vZM4IrQf4Ztu5eUAeCBUNR2GKxP4gq4zRw== X-Received: by 2002:a63:6c49:: with SMTP id h70mr766458pgc.368.1637002757322; Mon, 15 Nov 2021 10:59:17 -0800 (PST) Received: from bbox-1.mtv.corp.google.com ([2620:15c:211:201:7a54:8083:4365:b23d]) by smtp.gmail.com with ESMTPSA id c3sm11882941pfv.67.2021.11.15.10.59.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 10:59:16 -0800 (PST) From: Minchan Kim To: Andrew Morton Cc: Sergey Senozhatsky , linux-mm , LKML , Minchan Kim Subject: [PATCH v2 4/9] zsmalloc: introduce obj_allocated Date: Mon, 15 Nov 2021 10:59:04 -0800 Message-Id: <20211115185909.3949505-5-minchan@kernel.org> X-Mailer: git-send-email 2.34.0.rc1.387.gb447b232ab-goog In-Reply-To: <20211115185909.3949505-1-minchan@kernel.org> References: <20211115185909.3949505-1-minchan@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 35F901923 X-Stat-Signature: m5tkno1erxtfjzgb3e84fonrnexq3z11 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=G6MoOgbv; spf=pass (imf22.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-HE-Tag: 1637002758-17922 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The usage pattern for obj_to_head is to check whether the zpage is allocated or not. Thus, introduce obj_allocated. Signed-off-by: Minchan Kim --- mm/zsmalloc.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 7a14090e4a53..6ca130c0f7dc 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -877,13 +877,21 @@ static unsigned long handle_to_obj(unsigned long ha= ndle) return *(unsigned long *)handle; } =20 -static unsigned long obj_to_head(struct page *page, void *obj) +static bool obj_allocated(struct page *page, void *obj, unsigned long *p= handle) { + unsigned long handle; + if (unlikely(PageHugeObject(page))) { VM_BUG_ON_PAGE(!is_first_page(page), page); - return page->index; + handle =3D page->index; } else - return *(unsigned long *)obj; + handle =3D *(unsigned long *)obj; + + if (!(handle & OBJ_ALLOCATED_TAG)) + return false; + + *phandle =3D handle & ~OBJ_ALLOCATED_TAG; + return true; } =20 static inline int testpin_tag(unsigned long handle) @@ -1606,7 +1614,6 @@ static void zs_object_copy(struct size_class *class= , unsigned long dst, static unsigned long find_alloced_obj(struct size_class *class, struct page *page, int *obj_idx) { - unsigned long head; int offset =3D 0; int index =3D *obj_idx; unsigned long handle =3D 0; @@ -1616,9 +1623,7 @@ static unsigned long find_alloced_obj(struct size_c= lass *class, offset +=3D class->size * index; =20 while (offset < PAGE_SIZE) { - head =3D obj_to_head(page, addr + offset); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, addr + offset, &handle)) { if (trypin_tag(handle)) break; handle =3D 0; @@ -1928,7 +1933,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, struct page *dummy; void *s_addr, *d_addr, *addr; int offset, pos; - unsigned long handle, head; + unsigned long handle; unsigned long old_obj, new_obj; unsigned int obj_idx; int ret =3D -EAGAIN; @@ -1964,9 +1969,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, pos =3D offset; s_addr =3D kmap_atomic(page); while (pos < PAGE_SIZE) { - head =3D obj_to_head(page, s_addr + pos); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, s_addr + pos, &handle)) { if (!trypin_tag(handle)) goto unpin_objects; } @@ -1982,9 +1985,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, =20 for (addr =3D s_addr + offset; addr < s_addr + pos; addr +=3D class->size) { - head =3D obj_to_head(page, addr); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, addr, &handle)) { BUG_ON(!testpin_tag(handle)); =20 old_obj =3D handle_to_obj(handle); @@ -2029,9 +2030,7 @@ static int zs_page_migrate(struct address_space *ma= pping, struct page *newpage, unpin_objects: for (addr =3D s_addr + offset; addr < s_addr + pos; addr +=3D class->size) { - head =3D obj_to_head(page, addr); - if (head & OBJ_ALLOCATED_TAG) { - handle =3D head & ~OBJ_ALLOCATED_TAG; + if (obj_allocated(page, addr, &handle)) { BUG_ON(!testpin_tag(handle)); unpin_tag(handle); } --=20 2.34.0.rc1.387.gb447b232ab-goog