From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 513E7C4321A for ; Fri, 28 Jun 2019 16:37:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 298AF2086D for ; Fri, 28 Jun 2019 16:37:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="PfFFq5Ci" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726869AbfF1QhX (ORCPT ); Fri, 28 Jun 2019 12:37:23 -0400 Received: from bombadil.infradead.org ([198.137.202.133]:59238 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726605AbfF1QhX (ORCPT ); Fri, 28 Jun 2019 12:37:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=YfwGz5sOrImHIjcEF62FXobE7G43pUuN/X5aC3DDmls=; b=PfFFq5CiBic4IjGw2dl6jXqTD vowp+ZjV1mlIL0kYmw9pueRCoT/0bSZQnb5IdgR9EmSRbaHIqxJkiiej2vV9Hg5j4YAEImOdGngr8 BAHWmeFf6tTJa1fgBiaLQfrp5jelVvJuz+lZGqmHhLTOvNbBePcVTNF/fhM0xIYSWsgyxGeo3BlRS 63UB5Uisoxvd7AldZFqcxcsCaFUt13z2IU8o9DIcyyYRBcvSywP9mHc0V+FiENQcA6szowYP/tq3p UYJt8O0XntbXvcj0nx4KoMsNdW8gqItattZ0PHBGWXzg2nbJ9qxuPYNRh+cvcAtGvdygJtt7Ou1bN rS7GAkEwA==; Received: from willy by bombadil.infradead.org with local (Exim 4.92 #3 (Red Hat Linux)) id 1hgtsH-0004uV-Ck; Fri, 28 Jun 2019 16:37:21 +0000 Date: Fri, 28 Jun 2019 09:37:21 -0700 From: Matthew Wilcox To: Dan Williams Cc: linux-nvdimm , Jan Kara , stable , Robert Barror , Seema Pandit , linux-fsdevel , Linux Kernel Mailing List Subject: Re: [PATCH] filesystem-dax: Disable PMD support Message-ID: <20190628163721.GC4286@bombadil.infradead.org> References: <156159454541.2964018.7466991316059381921.stgit@dwillia2-desk3.amr.corp.intel.com> <20190627123415.GA4286@bombadil.infradead.org> <20190627195948.GB4286@bombadil.infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Thu, Jun 27, 2019 at 07:39:37PM -0700, Dan Williams wrote: > On Thu, Jun 27, 2019 at 12:59 PM Matthew Wilcox wrote: > > On Thu, Jun 27, 2019 at 12:09:29PM -0700, Dan Williams wrote: > > > > This bug feels like we failed to unlock, or unlocked the wrong entry > > > > and this hunk in the bisected commit looks suspect to me. Why do we > > > > still need to drop the lock now that the radix_tree_preload() calls > > > > are gone? > > > > > > Nevermind, unmapp_mapping_pages() takes a sleeping lock, but then I > > > wonder why we don't restart the lookup like the old implementation. > > > > If something can remove a locked entry, then that would seem like the > > real bug. Might be worth inserting a lookup there to make sure that it > > hasn't happened, I suppose? > > Nope, added a check, we do in fact get the same locked entry back > after dropping the lock. Okay, good, glad to have ruled that out. > The deadlock revolves around the mmap_sem. One thread holds it for > read and then gets stuck indefinitely in get_unlocked_entry(). Once > that happens another rocksdb thread tries to mmap and gets stuck > trying to take the mmap_sem for write. Then all new readers, including > ps and top that try to access a remote vma, then get queued behind > that write. > > It could also be the case that we're missing a wake up. That was the conclusion I came to; that one thread holding the mmap sem for read isn't being woken up when it should be. Just need to find it ... obviously it's something to do with the PMD entries.