From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F4D0C43331 for ; Tue, 12 Nov 2019 16:09:07 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7268720679 for ; Tue, 12 Nov 2019 16:09:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="JGddSZ/j" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7268720679 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9hb15Loo8OxT+yIX5ShLJZoMHVpRJC5l5/f/ITfrJjw=; b=JGddSZ/jSrfWf7 JgRckzAZIFks8AixNBo+jWHr6eR4hu9VfWewrMkx8IrlP3qDOyDj44UPIthnEIG1fSpcaNRvtD2Vn 24pV4eA3VKYczZ40hxeUSd2oIoxFduv9bPksuNGmrSi4/HBscV/mTvCYWkAToZ/ncGal4p4OCbbac 1mK0Y+wV4grFYUDq+AGrKWcvATNmB5mDBCSKSTybribq0qCcagKvIX+gVgfz8JPdPSF2DrmefVWKC 3VuU/mWr+okMGvPM8wkGZW+ZKSkHOU34tSfJmjy7XZgg7TZGTPX0fJIpvG+oi1oERwChVdZincB7z gLgloXLg45mIwjMfygGg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iUYj4-0002tJ-Vz; Tue, 12 Nov 2019 16:09:06 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iUYiz-0002sQ-CF for linux-arm-kernel@lists.infradead.org; Tue, 12 Nov 2019 16:09:05 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 639A3D6E; Tue, 12 Nov 2019 08:09:00 -0800 (PST) Received: from arrakis.emea.arm.com (arrakis.cambridge.arm.com [10.1.197.42]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 29C753F534; Tue, 12 Nov 2019 08:08:59 -0800 (PST) Date: Tue, 12 Nov 2019 16:08:57 +0000 From: Catalin Marinas To: Vincent Whitchurch Subject: Re: [PATCH v2] buffer: Fix I/O error due to ARM read-after-read hazard Message-ID: <20191112160855.GA22025@arrakis.emea.arm.com> References: <20191112130244.16630-1-vincent.whitchurch@axis.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20191112130244.16630-1-vincent.whitchurch@axis.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191112_080903_560057_65DFC8C9 X-CRM114-Status: GOOD ( 15.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: axboe@kernel.dk, Richard Earnshaw , Vincent Whitchurch , torvalds@linux-foundation.org, linux@armlinux.org.uk, linux-kernel@vger.kernel.org, will@kernel.org, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Nov 12, 2019 at 02:02:44PM +0100, Vincent Whitchurch wrote: > On my dual-core ARM Cortex-A9, reading from squashfs (over > dm-verity/ubi/mtd) in a loop for hundreds of hours invariably results in > a read failure in squashfs_read_data(). The errors occur because the > buffer_uptodate() check fails after wait_on_buffer(). Further debugging > shows that the bh was in fact uptodate and that there is no actual I/O > error in the lower layers. > > The problem is caused by the read-after-read hazards in the ARM > Cortex-A9 MPCore (erratum #761319, see [1]). The code generated by the > compiler for the combination of the wait_on_buffer() and > buffer_uptodate() calls reads the flags value twice from memory (see the > excerpt of the assembly below). The new value of the BH_Lock flag is > seen but the new value of BH_Uptodate is not even though both the bits > are read from the same memory location. > > 27c: 9d08 ldr r5, [sp, #32] > 27e: 2400 movs r4, #0 > 280: e006 b.n 290 > 282: 6803 ldr r3, [r0, #0] > 284: 07da lsls r2, r3, #31 > 286: f140 810d bpl.w 4a4 > 28a: 3401 adds r4, #1 > 28c: 42bc cmp r4, r7 > 28e: da08 bge.n 2a2 > 290: f855 0f04 ldr.w r0, [r5, #4]! > 294: 6803 ldr r3, [r0, #0] > 296: 0759 lsls r1, r3, #29 > 298: d5f3 bpl.n 282 > 29a: f7ff fffe bl 0 <__wait_on_buffer> > > Work around this problem by adding a DMB between the two reads of > bh->flags, as recommended in the ARM document. With this barrier, no > failures have been seen in more than 5000 hours of the same test. > > [1] http://infocenter.arm.com/help/topic/com.arm.doc.uan0004a/UAN0004A_a9_read_read.pdf I thought we were going to fix the compiler. I found an old thread here: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00714.html Also cc'ing Richard Earnshaw as he may been involved in the gcc discussion at the time. While you can add some barrier here, there may be other cases where this can go wrong. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel