From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.6 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A9B8C433E7 for ; Tue, 1 Sep 2020 12:58:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 146DC206EB for ; Tue, 1 Sep 2020 12:58:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727099AbgIAM62 (ORCPT ); Tue, 1 Sep 2020 08:58:28 -0400 Received: from mga11.intel.com ([192.55.52.93]:32279 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727035AbgIAM6Z (ORCPT ); Tue, 1 Sep 2020 08:58:25 -0400 IronPort-SDR: pqaaEbBqU9SHpuEC/fckcbaJa/4cEFW9o00SmTJGA+SU9ykw1a6/xPZmBPM6WB2wGduY6q9ZJB cC3VVA5QJSRQ== X-IronPort-AV: E=McAfee;i="6000,8403,9730"; a="154667439" X-IronPort-AV: E=Sophos;i="5.76,379,1592895600"; d="scan'208";a="154667439" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2020 05:58:21 -0700 IronPort-SDR: 4kXPbvOeXRtIxguQVgqQA1Cyw8GWOjZOztTfs9BSgyofckVG+BQ8kb3YqcjvKh6cirFkEcTy6t lRNb6VCU+Cbg== X-IronPort-AV: E=Sophos;i="5.76,379,1592895600"; d="scan'208";a="477189160" Received: from ddalessa-mobl.amr.corp.intel.com (HELO [10.254.202.100]) ([10.254.202.100]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2020 05:58:20 -0700 Subject: Re: buggy-looking mm_struct refcounting in HFI1 infiniband driver To: Jason Gunthorpe , Jann Horn Cc: Mike Marciniszyn , Ira Weiny , Linux-MM , Doug Ledford , linux-rdma@vger.kernel.org, Dean Luick References: <20200901002109.GG1152540@nvidia.com> From: Dennis Dalessandro Message-ID: <624472c4-b585-e950-78b2-eff860f24d64@intel.com> Date: Tue, 1 Sep 2020 08:58:18 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <20200901002109.GG1152540@nvidia.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On 8/31/2020 8:21 PM, Jason Gunthorpe wrote: > On Tue, Sep 01, 2020 at 01:45:06AM +0200, Jann Horn wrote: > >> struct hfi1_filedata has a member ->mm that holds a ->mm_count reference: >> >> static int hfi1_file_open(struct inode *inode, struct file *fp) >> { >> struct hfi1_filedata *fd; >> [...] >> fd->mm = current->mm; >> mmgrab(fd->mm); // increments ->mm_count >> [...] >> } > > Yikes, gross. > >> However, e.g. the call chain hfi1_file_ioctl() -> user_exp_rcv_setup() >> -> hfi1_user_exp_rcv_setup() -> pin_rcv_pages() -> >> hfi1_acquire_user_pages() -> pin_user_pages_fast() can end up >> traversing VMAs without holding any ->mm_users reference, as far as I >> can tell. That will probably result in kernel memory corruption if >> that races the wrong way with an exiting task (with the ioctl() call >> coming from a task whose ->mm is different from fd->mm). > > It looks like this path should be using current and storing the grab'd > mm in the tidbuf for later use by hfi1_release_user_pages() > > The only other use of file->mm is to setup a notifier, but this is > also under hfi1_user_exp_rcv_setup() so it should just use tidbuf->mm > == current anyhow. > > The pq->mm looks similar, looks like the pq should use current->mm, > and it sets up an old-style notifier, but I didn't check carefully if > all the call paths are linked back to an ioctl.. > > It doesn't make sense that a RDMA driver would do any page pinning > outside an ioctl, so it should always use current. I sort of recall a bug where we were trusting current and it wasn't correct. I'll have to see if I can dig up the details and figure out what's going on here. >> Disclaimer: I haven't actually tested this - I just stumbled over it >> while working on some other stuff, and I don't have any infiniband >> hardware to test with. So it might well be that I just missed an >> mmget_not_zero() somewhere, or something like that. > > It looks wrong to me too. > > Dennis? I'll look at it closer and either send a patch or an explanation. Thanks for bringing it to our attention Jann! -Denny