From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753197AbdCATV2 (ORCPT ); Wed, 1 Mar 2017 14:21:28 -0500 Received: from mx2.suse.de ([195.135.220.15]:42710 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752514AbdCATVH (ORCPT ); Wed, 1 Mar 2017 14:21:07 -0500 Date: Wed, 1 Mar 2017 20:19:43 +0100 From: Michal Hocko To: Robert Kudyba Cc: linux-kernel@vger.kernel.org Subject: Re: rsync: page allocation stalls in kernel 4.9.10 to a VessRAID NAS Message-ID: <20170301191942.GC24905@dhcp22.suse.cz> References: <20170228144045.GD26792@dhcp22.suse.cz> <3E4C7821-A93D-4956-A0E0-730BEC67C9F0@fordham.edu> <20170228151535.GE26792@dhcp22.suse.cz> <63A3D887-EEDA-46D2-AB59-D5955FC3D23D@fordham.edu> <20170228165638.GA27726@dhcp22.suse.cz> <20170301080604.GB1124@dhcp22.suse.cz> <20170301173625.GA20360@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed 01-03-17 13:06:57, Robert Kudyba wrote: [...] > Does this ever finish? Once there is no memory pressure > The output file seems to grow really quickly. Here are a few lines and > I can upload the file somewhere if you think it’d be helpful: [...] > kswapd0-52 [001] .... 90900.812734: mm_vmscan_lru_shrink_inactive: nid=0 nr_scanned=32 nr_reclaimed=32 priority=12 flags=RECLAIM_WB_FILE|RECLAIM_WB_ASYNC > kswapd0-52 [001] .... 90900.813762: mm_vmscan_lru_shrink_inactive: nid=0 nr_scanned=1 nr_reclaimed=0 priority=11 flags=RECLAIM_WB_FILE|RECLAIM_WB_ASYNC > kswapd0-52 [001] .... 90900.813928: mm_vmscan_lru_shrink_inactive: nid=0 nr_scanned=32 nr_reclaimed=32 priority=11 flags=RECLAIM_WB_FILE|RECLAIM_WB_ASYNC > kswapd0-52 [001] .... 90900.814823: mm_vmscan_lru_shrink_inactive: nid=0 nr_scanned=22 nr_reclaimed=22 priority=11 flags=RECLAIM_WB_FILE|RECLAIM_WB_ASYNC > kswapd0-52 [001] .... 90900.815514: mm_vmscan_lru_shrink_inactive: nid=0 nr_scanned=3 nr_reclaimed=0 priority=10 flags=RECLAIM_WB_FILE|RECLAIM_WB_ASYNC > kswapd0-52 [001] .... 90900.815643: mm_vmscan_lru_shrink_inactive: nid=0 nr_scanned=32 nr_reclaimed=32 priority=10 flags=RECLAIM_WB_FILE|RECLAIM_WB_ASYNC > kswapd0-52 [002] .... 91073.571793: mm_vmscan_lru_shrink_inactive: nid=0 nr_scanned=27 nr_reclaimed=27 priority=5 flags=RECLAIM_WB_FILE|RECLAIM_WB_ASYNC all those are from the kswapd (background memory reclaim). Which means that it doesn't catch any allocation which can stall for too long. Anyway the above tracepoint show that we are able to make some progress during the reclaim (nr_reclaimed > 0). So I suspect that this is indeed a large lowmem pressure and I do not see what we can do about that... > > The rest of the report is rather messed up but I assume that you simply > > do not have contiguous memory in the lowmem. This surely sounds like a > > 32b specific problem which is not reasonably fixable. > > Messed up as in unreadable? yeah 2 reports interleaved. -- Michal Hocko SUSE Labs