From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Subject: dma-mapping: clearing GFP_ZERO flag caused crashes of Ethernet on arc/hsdk board. From: Alexey Brodkin Message-Id: <1522185558.6917.7.camel@synopsys.com> Date: Tue, 27 Mar 2018 21:19:19 +0000 To: "andy.shevchenko@gmail.com" Cc: "linux-kernel@vger.kernel.org" , "Alexey.Brodkin@synopsys.com" , "jesper.nilsson@axis.com" , "Eugeniy.Paltsev@synopsys.com" , "hch@lst.de" , "linux-snps-arc@lists.infradead.org" , "iommu@lists.linux-foundation.org" , "dmaengine@vger.kernel.org" , "Evgeniy.Didin@synopsys.com" , "geert@linux-m68k.org" List-ID: SGkgQW5keSwNCg0KT24gVHVlLCAyMDE4LTAzLTI3IGF0IDIxOjExICswMzAwLCBBbmR5IFNoZXZj aGVua28gd3JvdGU6DQo+IE9uIFR1ZSwgTWFyIDI3LCAyMDE4IGF0IDg6MTIgUE0sIEV2Z2VuaXkg RGlkaW4NCj4gPEV2Z2VuaXkuRGlkaW5Ac3lub3BzeXMuY29tPiB3cm90ZToNCj4gPiBIZWxsbywN Cj4gPiANCj4gPiBBZnRlciBjb21taXQgIDU3YmY1YTg5NjNmOCAoImRtYS1tYXBwaW5nOiBjbGVh ciBoYXJtZnVsIEdGUF8qIGZsYWdzIGluIGNvbW1vbiBjb2RlIikgIHdlIG5vdGljZWQgcHJvYmxl bXMgd2l0aCBFdGhlcm5ldCBjb250cm9sbGVyIG9uIG9uZSBvZiBvdXINCj4gPiBwbGF0Zm9ybXMg KG5hbWVseSBBUkMgSFNESykuDQo+ID4gSQ0KPiA+IG4gcGFydGljdWxhciB3ZSBzZWUgdGhhdCBy ZW1vdmFsIG9mIF9fR0ZQX1pFUk8gZmxhZyBpbiBmdW5jdGlvbiBkbWFfYWxsb2NfYXR0cnMoKSB3 YXMgdGhlIGN1bHByaXQgYmVjYXVzZSBpbiBvdXIgaW1wbGVtZW50YXRpb24gb2YgYXJjX2RtYV9h bGxvYygpDQo+ID4gd2Ugb25seSBhbGxvY2F0ZSB6ZXJvZWQgcGFnZXMgaWYNCj4gPiB0aGF0IGZs YWcgaXMgZXhwbGljaXRseSBzZXQgYnkgdGhlIGNhbGxlci4gTm93IHdpdGggdW5jb25kaXRpb25h bCByZW1vdmFsIG9mIHRoYXQgZmxhZyBpbiBkbWFfYWxsb2NfYXR0cnMoKSB3ZSBhbGxvY2F0ZSBu b24temVyb2VkIHBhZ2VzIGFuZCB0aGF0DQo+ID4gc2VlbSB0byBjYXVzZSBwcm9ibGVtcy4NCj4g PiANCj4gPiBGcm9tDQo+ID4gbWVudGlvbmVkIGNvbW1pdCBtZXNzYWdlIEkgbWF5IGNvbmNsdWRl IHRoYXQgYXJjaGl0ZWN0dXJhbCBjb2RlIGlzIHN1cHBvc2VkIHRvIGFsd2F5cyBhbGxvY2F0ZSB6 ZXJvZWQgcGFnZXMgYnV0IEkgY2Fubm90IGZpbmQgYW55IHJlcXVpcmVtZW50IG9mDQo+ID4gdGhh dCBpbiBrZXJuZWwncyBkb2N1bWVudGF0aW9uLg0KPiA+IENvdWwNCj4gPiBkIHlvdSBwbGVhc2Ug cG9pbnQgbWUgdG8gdGhhdCByZXF1aXJlbWVudCBpZiB0aGF0IGV4aXN0cyBhdCBhbGwsIHRoZW4g d2UnbGwgaW1wbGVtZW50IGEgZml4IGluIG91ciBhcmNoIGNvZGUgbGlrZSB0aGF0Og0KPiANCj4g Q2FuIHlvdSBlbGFib3JhdGUgd2hhdCBkcml2ZXIgaXMgaW4gdXNlPw0KPiBzdG1tYWMgd2l0aCBk d21hYy1hbmFyaW9uPw0KDQpJdCBpcyBpbmRlZWQgRFcgR01BQyAoQUtBIFNUTU1BQykgd2l0aCBi dWlsdC1pbiBETUEuDQoNCj4gSWYgc28sIHRoaXMgZHJpdmVyICh3L28gYW5hcmlvbiBwYXJ0cywg d2hpY2ggSSBiZWxpZXZlIGRvZXNuJ3QgaGF2ZQ0KPiBhbnl0aGluZyB0byBkbyB3aXRoIHRoaXMp IGlzIHdpZGVseSB1c2VkIG9uIG90aGVyIHBsYXRmb3Jtcy4NCj4gV2UgaGF2ZSB0byBzZWUgYSBs b3Qgb2YgcmVwb3J0cywgdGhvdWdoIG9ubHkgb25lIHNvIGZhcj8NCj4gDQo+IFRoZSBsb2dpY2Fs IHF1ZXN0aW9uIGlzIHdoeT8NCg0KMS4gU2VlIHRoYXQncyBhbm90aGVyIHBsYXRmb3JtIHdpdGgg QVJDIGNvcmUgc28gbWF5YmUgaW4gY2FzZSBvZiBBUk0NCiAgIERNQSBhbGxvY2F0b3IgYWxyZWFk eSB6ZXJvZXMgcGFnZXMgcmVnYXJkbGVzcyBwcm92aWRlZCBmbGFncyAtDQogICBwZXJzb25hbGx5 IEkgZGlkbid0IGNoZWNrIHRoYXQuDQoNCjIuIEV2ZW4gb24gSFNESyB3ZSBzYXcgdGhhdCBvbmx5 IG9uIGF0dGVtcHQgdG8gcnVuICJpcGVyZiIsIGV2ZW4gREhDUA0KICAgY2xpZW50IHdvcmtzIHBl cmZlY3RseSBmaW5lIG9uIHRoYXQgc2FtZSBwbGF0Zm9ybSBzbyBtYXliZSBvdGhlcnMNCiAgIGp1 c3QgZG9uJ3Qgc2VlIHByb2JsZW1zIHlldC4NCg0KMy4gV2hvIGtub3dzIGlmIFJDcyBhcmUgYmVp bmcgdGVzdGVkIG9uIG90aGVyIHBsYXRmb3JtcyB3aXRoDQogICBuZXR3b3JraW5nIHNvIG1heWJl IHNpbWlsYXIgcmVwb3J0cyB3aWxsIHN0YXJ0IHRvIGFwcGVhciBvbmNlDQogICA0LjE2IGdldHMg cmVsZWFzZWQuDQoNCi1BbGV4ZXkK From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexey Brodkin Subject: Re: dma-mapping: clearing GFP_ZERO flag caused crashes of Ethernet on arc/hsdk board. Date: Tue, 27 Mar 2018 21:19:19 +0000 Message-ID: <1522185558.6917.7.camel@synopsys.com> References: <1522170774.2593.9.camel@synopsys.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Content-Language: en-US Content-ID: <64D7FDC862E9D24C95389F44BE802964-z7JfP6tgrtVBCHUSTMH8dZqQE7yCjDx5@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: "andy.shevchenko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org" Cc: "jesper.nilsson-VrBV9hrLPhE@public.gmane.org" , "Alexey.Brodkin-HKixBCOQz3hWk0Htik3J/w@public.gmane.org" , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "Evgeniy.Didin-HKixBCOQz3hWk0Htik3J/w@public.gmane.org" , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , "geert-Td1EMuHUCqxL1ZNQvxDV9g@public.gmane.org" , "dmaengine-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-snps-arc-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" , "Eugeniy.Paltsev-HKixBCOQz3hWk0Htik3J/w@public.gmane.org" , "hch-jcswGhMUV9g@public.gmane.org" List-Id: iommu@lists.linux-foundation.org Hi Andy, On Tue, 2018-03-27 at 21:11 +0300, Andy Shevchenko wrote: > On Tue, Mar 27, 2018 at 8:12 PM, Evgeniy Didin > wrote: > > Hello, > > > > After commit 57bf5a8963f8 ("dma-mapping: clear harmful GFP_* flags in common code") we noticed problems with Ethernet controller on one of our > > platforms (namely ARC HSDK). > > I > > n particular we see that removal of __GFP_ZERO flag in function dma_alloc_attrs() was the culprit because in our implementation of arc_dma_alloc() > > we only allocate zeroed pages if > > that flag is explicitly set by the caller. Now with unconditional removal of that flag in dma_alloc_attrs() we allocate non-zeroed pages and that > > seem to cause problems. > > > > From > > mentioned commit message I may conclude that architectural code is supposed to always allocate zeroed pages but I cannot find any requirement of > > that in kernel's documentation. > > Coul > > d you please point me to that requirement if that exists at all, then we'll implement a fix in our arch code like that: > > Can you elaborate what driver is in use? > stmmac with dwmac-anarion? It is indeed DW GMAC (AKA STMMAC) with built-in DMA. > If so, this driver (w/o anarion parts, which I believe doesn't have > anything to do with this) is widely used on other platforms. > We have to see a lot of reports, though only one so far? > > The logical question is why? 1. See that's another platform with ARC core so maybe in case of ARM DMA allocator already zeroes pages regardless provided flags - personally I didn't check that. 2. Even on HSDK we saw that only on attempt to run "iperf", even DHCP client works perfectly fine on that same platform so maybe others just don't see problems yet. 3. Who knows if RCs are being tested on other platforms with networking so maybe similar reports will start to appear once 4.16 gets released. -Alexey From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexey.Brodkin@synopsys.com (Alexey Brodkin) Date: Tue, 27 Mar 2018 21:19:19 +0000 Subject: dma-mapping: clearing GFP_ZERO flag caused crashes of Ethernet on arc/hsdk board. In-Reply-To: References: <1522170774.2593.9.camel@synopsys.com> List-ID: Message-ID: <1522185558.6917.7.camel@synopsys.com> To: linux-snps-arc@lists.infradead.org Hi Andy, On Tue, 2018-03-27@21:11 +0300, Andy Shevchenko wrote: > On Tue, Mar 27, 2018 at 8:12 PM, Evgeniy Didin > wrote: > > Hello, > > > > After commit 57bf5a8963f8 ("dma-mapping: clear harmful GFP_* flags in common code") we noticed problems with Ethernet controller on one of our > > platforms (namely ARC HSDK). > > I > > n particular we see that removal of __GFP_ZERO flag in function dma_alloc_attrs() was the culprit because in our implementation of arc_dma_alloc() > > we only allocate zeroed pages if > > that flag is explicitly set by the caller. Now with unconditional removal of that flag in dma_alloc_attrs() we allocate non-zeroed pages and that > > seem to cause problems. > > > > From > > mentioned commit message I may conclude that architectural code is supposed to always allocate zeroed pages but I cannot find any requirement of > > that in kernel's documentation. > > Coul > > d you please point me to that requirement if that exists at all, then we'll implement a fix in our arch code like that: > > Can you elaborate what driver is in use? > stmmac with dwmac-anarion? It is indeed DW GMAC (AKA STMMAC) with built-in DMA. > If so, this driver (w/o anarion parts, which I believe doesn't have > anything to do with this) is widely used on other platforms. > We have to see a lot of reports, though only one so far? > > The logical question is why? 1. See that's another platform with ARC core so maybe in case of ARM DMA allocator already zeroes pages regardless provided flags - personally I didn't check that. 2. Even on HSDK we saw that only on attempt to run "iperf", even DHCP client works perfectly fine on that same platform so maybe others just don't see problems yet. 3. Who knows if RCs are being tested on other platforms with networking so maybe similar reports will start to appear once 4.16 gets released. -Alexey From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752096AbeC0VTd (ORCPT ); Tue, 27 Mar 2018 17:19:33 -0400 Received: from us01smtprelay-2.synopsys.com ([198.182.60.111]:44073 "EHLO smtprelay.synopsys.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751008AbeC0VTb (ORCPT ); Tue, 27 Mar 2018 17:19:31 -0400 From: Alexey Brodkin To: "andy.shevchenko@gmail.com" CC: "linux-kernel@vger.kernel.org" , "Alexey.Brodkin@synopsys.com" , "jesper.nilsson@axis.com" , "Eugeniy.Paltsev@synopsys.com" , "hch@lst.de" , "linux-snps-arc@lists.infradead.org" , "iommu@lists.linux-foundation.org" , "dmaengine@vger.kernel.org" , "Evgeniy.Didin@synopsys.com" , "geert@linux-m68k.org" Subject: Re: dma-mapping: clearing GFP_ZERO flag caused crashes of Ethernet on arc/hsdk board. Thread-Topic: dma-mapping: clearing GFP_ZERO flag caused crashes of Ethernet on arc/hsdk board. Thread-Index: AQHTxe7Y86AnIPeLoEWcNLwKgl2z76PkQKwAgAA0dwA= Date: Tue, 27 Mar 2018 21:19:19 +0000 Message-ID: <1522185558.6917.7.camel@synopsys.com> References: <1522170774.2593.9.camel@synopsys.com> In-Reply-To: Accept-Language: en-US, ru-RU Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.225.15.87] Content-Type: text/plain; charset="utf-8" Content-ID: <64D7FDC862E9D24C95389F44BE802964@internal.synopsys.com> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id w2RLJdhw005946 Hi Andy, On Tue, 2018-03-27 at 21:11 +0300, Andy Shevchenko wrote: > On Tue, Mar 27, 2018 at 8:12 PM, Evgeniy Didin > wrote: > > Hello, > > > > After commit 57bf5a8963f8 ("dma-mapping: clear harmful GFP_* flags in common code") we noticed problems with Ethernet controller on one of our > > platforms (namely ARC HSDK). > > I > > n particular we see that removal of __GFP_ZERO flag in function dma_alloc_attrs() was the culprit because in our implementation of arc_dma_alloc() > > we only allocate zeroed pages if > > that flag is explicitly set by the caller. Now with unconditional removal of that flag in dma_alloc_attrs() we allocate non-zeroed pages and that > > seem to cause problems. > > > > From > > mentioned commit message I may conclude that architectural code is supposed to always allocate zeroed pages but I cannot find any requirement of > > that in kernel's documentation. > > Coul > > d you please point me to that requirement if that exists at all, then we'll implement a fix in our arch code like that: > > Can you elaborate what driver is in use? > stmmac with dwmac-anarion? It is indeed DW GMAC (AKA STMMAC) with built-in DMA. > If so, this driver (w/o anarion parts, which I believe doesn't have > anything to do with this) is widely used on other platforms. > We have to see a lot of reports, though only one so far? > > The logical question is why? 1. See that's another platform with ARC core so maybe in case of ARM DMA allocator already zeroes pages regardless provided flags - personally I didn't check that. 2. Even on HSDK we saw that only on attempt to run "iperf", even DHCP client works perfectly fine on that same platform so maybe others just don't see problems yet. 3. Who knows if RCs are being tested on other platforms with networking so maybe similar reports will start to appear once 4.16 gets released. -Alexey