From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1057C0015E for ; Tue, 25 Jul 2023 11:16:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D478E8D0001; Tue, 25 Jul 2023 07:16:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF6C86B0074; Tue, 25 Jul 2023 07:16:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE5D98D0001; Tue, 25 Jul 2023 07:16:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B0FE36B0071 for ; Tue, 25 Jul 2023 07:16:38 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6BDB2B25D4 for ; Tue, 25 Jul 2023 11:16:38 +0000 (UTC) X-FDA: 81049881276.15.45A8D9A Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) by imf02.hostedemail.com (Postfix) with ESMTP id 63F2380019 for ; Tue, 25 Jul 2023 11:16:35 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf02.hostedemail.com: domain of f.ebner@proxmox.com designates 94.136.29.106 as permitted sender) smtp.mailfrom=f.ebner@proxmox.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690283796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references; bh=NPuRthGWakbS6/h4t5EM8I8PmcQRIlx/Kgwpx80RfQI=; b=gCJMQLfWDXf3IJJ0iKBLFHmorwAY39aCcsfVi8OfHEc3rD/h357V72fpolkhoJAnaqEyD2 WK9kIVv8wBJjVoYZaSbFtaRyxxN9OcQyL3vS8DNhsiuEVMF7U1gtL2yAa276Bt0m70wd+F TkK/k+CHF2EUpuoybJKerSUMhUw/V+I= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf02.hostedemail.com: domain of f.ebner@proxmox.com designates 94.136.29.106 as permitted sender) smtp.mailfrom=f.ebner@proxmox.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690283796; a=rsa-sha256; cv=none; b=SVdjb02bBEICUyxFtlYJe75T82nkeOZE5/ibnKcRhqsKXM+7ye/lCXcs0a/iZP77U1lRt7 piFrMdhnkJ4H661715OVqjwjujX05547Cq831SC1eIyn9jSfC/RLBdv+0B9Q2T/XuldEFd cfJKwouGoK1Fh2EqggSZxXCjPB1nI5Q= Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 183F143CB5; Tue, 25 Jul 2023 13:16:33 +0200 (CEST) Message-ID: <8d063a26-43f5-0bb7-3203-c6a04dc159f8@proxmox.com> Date: Tue, 25 Jul 2023 13:16:27 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Content-Language: en-US From: Fiona Ebner To: torvalds@linux-foundation.org, akpm@linux-foundation.org Cc: Thomas Lamprecht , Wolfgang Bumiller , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: segfaults of processes while being killed after commit "mm: make the page fault mmap locking killable" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 63F2380019 X-Stat-Signature: 9bpw6ibfs9im6apy51nynwz3jtw7z69p X-HE-Tag: 1690283795-765763 X-HE-Meta: U2FsdGVkX18DVrQL72jjPtJnoIj4+DCLbC1w4HxRlkQNPqAKQ1rkyzutFjCGlwhGeIPPP+AUSF0THaHip75nvFNSpq2zv6e8fUvjLOsOWhEtBAnjF6ghnu8CKfiIQT2BSl43U/D7dh7gMpbuxr6v9m1fnXHXxpj9ZV4sTRzIGqghTRw+Jb1oj44M8Vjzl8e60DDwybOdkZLmk4u+1GNxbkpfNOq38dXomFWP14PDuvhtOzU3vpMwzRDYogAb1WRU9vNjHV0UtAG1G3XbHUaQCKPtL8K7e2O9XsXqZ32LswYEyKzhNy5g0jaNDp0TW3z+hEDsikKeBo8ejLBHHZQ7mKXM8kqhR/a1t1YDavxyur+q/tWXotwz9EPA24JX3vGKVHyg7eSUR5M5lUd1vNSrpdciL77tXpt9hj0u1TXKX2us7scU0HI7hwcxA/mxIrElvj6TJyoZiSIuY0/CXlArCu0MU3B471xyvXdTmwvj10+YEYHZ15K3ZFG0+IRzNiv4Yc0meGqBws60u9plTCotQXG5NDyzNbdWrAGOs7ugQjZp8PzAqGQMCIO8uXzyhVA0Vj4XdwZAafEGR4aJeAnCuc8LfkYZVQLawo8CLIRBsPnjdwexpqs/sXu3v8whZZo9lEyw0FG4Aq4oNs8E66lLuRL+SOUae8jOXuv4iXMBXZxtdxbvKqfiK8re9aEqQSxE8EXBbc4bOLuaFRF9PWE7WQCFZkvb2Sk4akD9aRwG1Q5t0XlgzfVSYNgPoJSJEuzdMU1XRdfXBizJjcCUyX82BXI3pWQFT4Q9ekWBBn/22NaaCaGFxxkzqtrBYBqSLKSX2aFxPRImGbgJa55qcTyUnek309hKs+UHZjZHWXf48sQekpFHP0w3/YhKQeViiKO/a1U1TWlRjp6bh4naMqSOHKXNNUvk8fjArJhbuDGX7WfrktfnNXdpPeOnf2AvNqO4PWgiYKiMalhaJpb13Xb GpOZpLMf e9Vxjw9eXKMbGMQC4tvcVVwMmWw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, we are seeing segfaults from processes while being killed starting with kernels which include commit eda0047296a16d65a7f2bc60a408f70d178b2014 ("mm: make the page fault mmap locking killable") all the way up to v6.5-rc3 which is the kernel I based this report on. I don't have a simple reproducer unfortunately, the one I have is big and quite racy. My working theory for what happens is (see also the bpftrace script and output [0]): Since get_mmap_lock_carefully() now uses mmap_read_lock_killable(), if rwsem_down_write_slowpath() is taken and there is a fatal signal pending, rwsem_down_write_slowpath() will return -EINTR and this is propagated up until get_mmap_lock_carefully() will return its boolean negation with !mmap_read_lock_killable(mm), i.e. 0. Then lock_mm_and_find_vma() returns NULL > if (!get_mmap_lock_carefully(mm, regs)) > return NULL; and so do_user_addr_fault() > vma = lock_mm_and_find_vma(mm, address, regs); > if (unlikely(!vma)) { > bad_area_nosemaphore(regs, error_code, address); > return; > } will end up without a vma and cause/log the segfault. Of course the process is already being killed, but I'd argue it is very confusing to users when apparent segfaults from such processes are being logged by the kernel. Happy to provide other traces or information if required! Best Regards, Fiona [0]: I ended up with the following bpftrace script > #include > #include > > kprobe:down_read_killable { > printf("%s %d %d\n", func, pid, tid); > } > > kprobe:rwsem_down_read_slowpath { > printf("%s %d %d\n", func, pid, tid); > } > > kretprobe:rwsem_down_read_slowpath { > printf("%s %d %d retval 0x%x\n", func, pid, tid, retval); > printf("%s\n", kstack()); > } > > kprobe:bad_area_nosemaphore { > printf("%s %d %d %s pending signal: %d\n", func, pid, tid, comm, > curtask->pending.signal.sig[0] > ); > if (curtask->pending.signal.sig[0]) { > printf("%s\n", kstack()); > } > } and here is a capture of a process running into the segfault. AFAIU, the pending signal translates to SIGKILL and the return value from down_read_killable() is -EINTR. > down_read_killable 987299 987299 > rwsem_down_read_slowpath 987299 987299 > down_read_killable 987299 987299 retval 0xfffffffc > > down_read_killable+72 > lock_mm_and_find_vma+167 > do_user_addr_fault+477 > exc_page_fault+131 > asm_exc_page_fault+39 > > bad_area_nosemaphore 987299 987299 pverados pending signal: 256 > > bad_area_nosemaphore+1 > exc_page_fault+131 > asm_exc_page_fault+39 > > bad_area_nosemaphore 987299 987299 pverados pending signal: 256 > > bad_area_nosemaphore+1 > exc_page_fault+131 > asm_exc_page_fault+39 > rep_movs_alternative+96 > show_opcodes+118 > __bad_area_nosemaphore+640 > bad_area_nosemaphore+22 > do_user_addr_fault+708 > exc_page_fault+131 > asm_exc_page_fault+39 > > bad_area_nosemaphore 987299 987299 pverados pending signal: 256 > > bad_area_nosemaphore+1 > exc_page_fault+131 > asm_exc_page_fault+39 > rep_movs_alternative+15 > show_opcodes+118 > __bad_area_nosemaphore+640 > bad_area_nosemaphore+22 > do_user_addr_fault+708 > exc_page_fault+131 > asm_exc_page_fault+39 >