From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E21AC433EF for ; Mon, 13 Jun 2022 14:48:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CEDA38D0191; Mon, 13 Jun 2022 10:48:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9E128D0190; Mon, 13 Jun 2022 10:48:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B16418D0191; Mon, 13 Jun 2022 10:48:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A26D78D0190 for ; Mon, 13 Jun 2022 10:48:43 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7EC4E32E86 for ; Mon, 13 Jun 2022 14:48:43 +0000 (UTC) X-FDA: 79573494126.24.380C07E Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) by imf26.hostedemail.com (Postfix) with ESMTP id 513E5140122 for ; Mon, 13 Jun 2022 14:48:11 +0000 (UTC) Received: by mail-lj1-f175.google.com with SMTP id e4so6490361ljl.1 for ; Mon, 13 Jun 2022 07:48:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=eTQgrOtNKXeuE24E4z1BZTvFXtuq4jZXbc2fnCBYhC4=; b=Nj4MPAUCb8OsNfEZnsGGbUCGhFTknfSsE2F2y/SD3ys8pENJtRCBqv0QGE5MeJqZKf 4ujqAiYBhiFAQIIMPaygZHU0/g+3YknfQZZNi4Zk1JY22jEvvgIxbsyGHQl/L5VMCTv+ PhibmZ6Q8zOwiyhaUUxlLced1et9/+osLVbICikxeqZICFXFHZtDonOd+BrKN251SWnv dxbwERu7dgYthvRMX8FfmxERfODU787Zqp6sW7mpxaFLu2txINVkCxHxdrUWAlAktt3B 5jBGTrYKi//rCcqiMtbP1tfziE+slRshLlGSrrgWkutmG5+D2MFut4UwQ9MF72FkL5M7 IRPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=eTQgrOtNKXeuE24E4z1BZTvFXtuq4jZXbc2fnCBYhC4=; b=i4toV2VXRbJ59P4gSUenMXtN7FkYC5lamRibwinQkpEWZhIKK+DZ4mumgbf9rmnsLX ezHnbreO1Bf9TRdvq0xG74afexKAIJjMXNOZqBSJnO0Nms5ZeTEsqGoLwZFC5Uw0ENf5 khfsvY6ZwaSmJLfXPcy1AC8phhjOJ91JFG/SXcF5Tm5bywLMYnAldXxFSapLSJzFAYnu HeKaLJCMT2xoLAe9CvfofRvDcTISSwLMA7Q+T5LZtAn0sHm15KLA4wVvpRpgDEERWmdw 7p+pZL7Ni7CbbUjZsaij6YnP36i9uWKYUi7q34L9BKwC7BG2veotUrdEt+HgwrbnIfvD G/Xw== X-Gm-Message-State: AOAM531/PqGxs4TRpGJAkfaolkzyU7OfCLBL92Q3MBtOcaOhNwmRPuxq q+J/wF4SnrhzJwGqyEcpKSvbMty9x1w= X-Google-Smtp-Source: ABdhPJyT0uwBN5ukf/MuNShwizt1LoZp0+Oby2zBTUXp3fSO81I8QLT2dBcsK6cZgTnu+DBQ4P/AYA== X-Received: by 2002:a05:6402:1e92:b0:42d:dc34:e233 with SMTP id f18-20020a0564021e9200b0042ddc34e233mr65598749edf.386.1655124957390; Mon, 13 Jun 2022 05:55:57 -0700 (PDT) Received: from [192.168.178.21] (p57b0b659.dip0.t-ipconnect.de. [87.176.182.89]) by smtp.gmail.com with ESMTPSA id v20-20020a1709060b5400b0070662b3b792sm3765548ejg.222.2022.06.13.05.55.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 13 Jun 2022 05:55:56 -0700 (PDT) Message-ID: <34daa8ab-a9f4-8f7b-0ea7-821bc36b9497@gmail.com> Date: Mon, 13 Jun 2022 14:55:54 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: [PATCH 03/13] mm: shmem: provide oom badness for shmem files Content-Language: en-US To: Michal Hocko , =?UTF-8?Q?Christian_K=c3=b6nig?= Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, intel-gfx@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-tegra@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, alexander.deucher@amd.com, daniel@ffwll.ch, viro@zeniv.linux.org.uk, akpm@linux-foundation.org, hughd@google.com, andrey.grodzovsky@amd.com References: <3f7d3d96-0858-fb6d-07a3-4c18964f888e@gmail.com> <2e7e050e-04eb-0c0a-0675-d7f1c3ae7aed@amd.com> <288528c3-411e-fb25-2f08-92d4bb9f1f13@gmail.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655131690; a=rsa-sha256; cv=none; b=a+iUw+yBY2HCgD7xJ2vWeBOU0Zhijc91+hn1DiYkivuE/D/BEK5pY1gTs09620P3yKaim/ uNu+4uzmyc7XVRLgrJEI1eMmQeKl0lKQ4P711faY//7BYn06UH4Vu2htrbh1nq1BNuI8Pc Msm85WfeOzhCpoCu10kjuzaUJXwqyDA= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Nj4MPAUC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of ckoenig.leichtzumerken@gmail.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=ckoenig.leichtzumerken@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655131690; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eTQgrOtNKXeuE24E4z1BZTvFXtuq4jZXbc2fnCBYhC4=; b=cBDHJ1tXgKIhDLZFJq3hhJrLejUAfwJR464DbI/TIjfEmp7pmEgzbBpeADc2GG01eYZFvA FgBbfYSrX6FIR9NvRHvswMiTb2Oft5QLfB4B3bQbyRHSegRXNYC5iNeXuz9jpoWPgggiKF 3PsrH2r8KHBO2owRVihCAPM8xI5/fiU= Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Nj4MPAUC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of ckoenig.leichtzumerken@gmail.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=ckoenig.leichtzumerken@gmail.com X-Rspam-User: X-Rspamd-Server: rspam12 X-Stat-Signature: skxpt787xxhmft73typjqn9ymdxp8wz9 X-Rspamd-Queue-Id: 513E5140122 X-HE-Tag: 1655131691-330080 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Am 13.06.22 um 14:11 schrieb Michal Hocko: > [SNIP] >>>> Alternative I could try to track the "owner" of a buffer (e.g. a shmem >>>> file), but then it can happen that one processes creates the object and >>>> another one is writing to it and actually allocating the memory. >>> If you can enforce that the owner is really responsible for the >>> allocation then all should be fine. That would require MAP_POPULATE like >>> semantic and I suspect this is not really feasible with the existing >>> userspace. It would be certainly hard to enforce for bad players. >> I've tried this today and the result was: "BUG: Bad rss-counter state >> mm:000000008751d9ff type:MM_FILEPAGES val:-571286". >> >> The problem is once more that files are not informed when the process >> clones. So what happened is that somebody called fork() with an mm_struct >> I've accounted my pages to. The result is just that we messed up the >> rss_stats and  the the "BUG..." above. >> >> The key difference between normal allocated pages and the resources here is >> just that we are not bound to an mm_struct in any way. > It is not really clear to me what exactly you have tried. I've tried to track the "owner" of a driver connection by keeping a reference to the mm_struct who created this connection inside our file private and then use add_mm_counter() to account all the allocations of the driver to this mm_struct. This works to the extend that now the right process is killed in an OOM situation. The problem with this approach is that the driver is not informed about operations like fork() or clone(), so what happens is that after a fork()/clone() we have an unbalanced rss-counter. Let me maybe get back to the initial question: We have resources which are not related to the virtual address space of a process, how should we tell the OOM killer about them? Thanks for all the input so far, Christian.