From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f44.google.com (mail-pb0-f44.google.com [209.85.160.44]) by kanga.kvack.org (Postfix) with ESMTP id 654446B0108 for ; Tue, 18 Mar 2014 12:33:33 -0400 (EDT) Received: by mail-pb0-f44.google.com with SMTP id rp16so7563704pbb.3 for ; Tue, 18 Mar 2014 09:33:33 -0700 (PDT) Received: from mga09.intel.com (mga09.intel.com. [134.134.136.24]) by mx.google.com with ESMTP id bs8si7506092pad.94.2014.03.18.09.33.31 for ; Tue, 18 Mar 2014 09:33:32 -0700 (PDT) Message-ID: <5328753B.2050107@intel.com> Date: Tue, 18 Mar 2014 09:32:59 -0700 From: Dave Hansen MIME-Version: 1.0 Subject: [LSF/MM TOPIC] Testing Large-Memory Hardware Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Linux-MM , LKML , lsf@lists.linux-foundation.org, Wu Fengguang I have a quick topic that could perhaps be addressed along with the testing topic that Dave Jones proposed. I won't be attending, but there will be a couple of other Intel folks there. This should be a fairly quick thing to address. Topic: Fengguang Wu who runs the wonderful LKP and 0day build tests was recently asking if I thought there was value in adding a large-memory system, say with 1TB of RAM. LKP is the system that generates these kinds of automated bug reports and performance tests: http://lkml.org/lkml/2014/3/9/201 My gut reaction was that we'd probably be better served by putting resources in to systems with higher core counts rather than lots of RAM. I have encountered the occasional boot bug on my 1TB system, but it's far from a frequent occurrence, and even more infrequent to encounter things at runtime. Would folks agree with that? What kinds of tests, benchmarks, stress tests, etc... do folks run that are both valuable and can only be run on a system with a large amount of actual RAM? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ig0-f177.google.com (mail-ig0-f177.google.com [209.85.213.177]) by kanga.kvack.org (Postfix) with ESMTP id 590DE6B010C for ; Tue, 18 Mar 2014 12:51:05 -0400 (EDT) Received: by mail-ig0-f177.google.com with SMTP id ur14so9091848igb.4 for ; Tue, 18 Mar 2014 09:51:05 -0700 (PDT) Received: from merlin.infradead.org (merlin.infradead.org. [2001:4978:20e::2]) by mx.google.com with ESMTPS id pe7si22639859icc.6.2014.03.18.09.51.03 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Mar 2014 09:51:03 -0700 (PDT) Date: Tue, 18 Mar 2014 17:50:59 +0100 From: Peter Zijlstra Subject: Re: [Lsf] [LSF/MM TOPIC] Testing Large-Memory Hardware Message-ID: <20140318165059.GI22095@laptop.programming.kicks-ass.net> References: <5328753B.2050107@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5328753B.2050107@intel.com> Sender: owner-linux-mm@kvack.org List-ID: To: Dave Hansen Cc: Linux-MM , LKML , lsf@lists.linux-foundation.org, Wu Fengguang On Tue, Mar 18, 2014 at 09:32:59AM -0700, Dave Hansen wrote: > I have a quick topic that could perhaps be addressed along with the > testing topic that Dave Jones proposed. I won't be attending, but there > will be a couple of other Intel folks there. This should be a fairly > quick thing to address. > > Topic: > > Fengguang Wu who runs the wonderful LKP and 0day build tests was > recently asking if I thought there was value in adding a large-memory > system, say with 1TB of RAM. LKP is the system that generates these > kinds of automated bug reports and performance tests: > > http://lkml.org/lkml/2014/3/9/201 > > My gut reaction was that we'd probably be better served by putting > resources in to systems with higher core counts rather than lots of RAM. > I have encountered the occasional boot bug on my 1TB system, but it's > far from a frequent occurrence, and even more infrequent to encounter > things at runtime. > > Would folks agree with that? What kinds of tests, benchmarks, stress > tests, etc... do folks run that are both valuable and can only be run on > a system with a large amount of actual RAM? We had a sched-numa + kvm fail on really large systems the other day, but yeah in general such problems tend to be rare. Then again, without test coverage they will always be rare, for even if there were problems, nobody would notice :-) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f176.google.com (mail-pd0-f176.google.com [209.85.192.176]) by kanga.kvack.org (Postfix) with ESMTP id C4BE46B010E for ; Tue, 18 Mar 2014 13:26:43 -0400 (EDT) Received: by mail-pd0-f176.google.com with SMTP id r10so7369621pdi.21 for ; Tue, 18 Mar 2014 10:26:43 -0700 (PDT) Received: from qmta07.emeryville.ca.mail.comcast.net (qmta07.emeryville.ca.mail.comcast.net. [2001:558:fe2d:43:76:96:30:64]) by mx.google.com with ESMTP id xk4si11775583pbc.5.2014.03.18.10.26.42 for ; Tue, 18 Mar 2014 10:26:42 -0700 (PDT) Date: Tue, 18 Mar 2014 12:26:40 -0500 (CDT) From: Christoph Lameter Subject: Re: [Lsf] [LSF/MM TOPIC] Testing Large-Memory Hardware In-Reply-To: <20140318165059.GI22095@laptop.programming.kicks-ass.net> Message-ID: References: <5328753B.2050107@intel.com> <20140318165059.GI22095@laptop.programming.kicks-ass.net> Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Peter Zijlstra Cc: Dave Hansen , lsf@lists.linux-foundation.org, Linux-MM , Wu Fengguang , LKML On Tue, 18 Mar 2014, Peter Zijlstra wrote: > > My gut reaction was that we'd probably be better served by putting > > resources in to systems with higher core counts rather than lots of RAM. > > I have encountered the occasional boot bug on my 1TB system, but it's > > far from a frequent occurrence, and even more infrequent to encounter > > things at runtime. > > > > Would folks agree with that? What kinds of tests, benchmarks, stress > > tests, etc... do folks run that are both valuable and can only be run on > > a system with a large amount of actual RAM? > > We had a sched-numa + kvm fail on really large systems the other day, > but yeah in general such problems tend to be rare. Then again, without > test coverage they will always be rare, for even if there were problems, > nobody would notice :-) SGI had systems out there up to few PB of RAM. There were a couple of tricks to get this going. Bootup time was pretty long. I/O has to be done carefully. The MM subsystem used to work with these sizes (I have not had a chance to verify that recently). This was Itanium with 64K page size so you had a factor of 16 less page structs to process. What I saw there is one of the reasons why I would like to see larger page support in the kernel. Managing massive amounts of 4k pages is creation far too much overhead. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755829AbaCRQdd (ORCPT ); Tue, 18 Mar 2014 12:33:33 -0400 Received: from mga09.intel.com ([134.134.136.24]:4493 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755413AbaCRQdb (ORCPT ); Tue, 18 Mar 2014 12:33:31 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,678,1389772800"; d="scan'208";a="502403359" Message-ID: <5328753B.2050107@intel.com> Date: Tue, 18 Mar 2014 09:32:59 -0700 From: Dave Hansen User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Linux-MM , LKML , lsf@lists.linux-foundation.org, Wu Fengguang Subject: [LSF/MM TOPIC] Testing Large-Memory Hardware Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I have a quick topic that could perhaps be addressed along with the testing topic that Dave Jones proposed. I won't be attending, but there will be a couple of other Intel folks there. This should be a fairly quick thing to address. Topic: Fengguang Wu who runs the wonderful LKP and 0day build tests was recently asking if I thought there was value in adding a large-memory system, say with 1TB of RAM. LKP is the system that generates these kinds of automated bug reports and performance tests: http://lkml.org/lkml/2014/3/9/201 My gut reaction was that we'd probably be better served by putting resources in to systems with higher core counts rather than lots of RAM. I have encountered the occasional boot bug on my 1TB system, but it's far from a frequent occurrence, and even more infrequent to encounter things at runtime. Would folks agree with that? What kinds of tests, benchmarks, stress tests, etc... do folks run that are both valuable and can only be run on a system with a large amount of actual RAM? From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756563AbaCRQvJ (ORCPT ); Tue, 18 Mar 2014 12:51:09 -0400 Received: from merlin.infradead.org ([205.233.59.134]:38224 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754194AbaCRQvI (ORCPT ); Tue, 18 Mar 2014 12:51:08 -0400 Date: Tue, 18 Mar 2014 17:50:59 +0100 From: Peter Zijlstra To: Dave Hansen Cc: Linux-MM , LKML , lsf@lists.linux-foundation.org, Wu Fengguang Subject: Re: [Lsf] [LSF/MM TOPIC] Testing Large-Memory Hardware Message-ID: <20140318165059.GI22095@laptop.programming.kicks-ass.net> References: <5328753B.2050107@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5328753B.2050107@intel.com> User-Agent: Mutt/1.5.21 (2012-12-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 18, 2014 at 09:32:59AM -0700, Dave Hansen wrote: > I have a quick topic that could perhaps be addressed along with the > testing topic that Dave Jones proposed. I won't be attending, but there > will be a couple of other Intel folks there. This should be a fairly > quick thing to address. > > Topic: > > Fengguang Wu who runs the wonderful LKP and 0day build tests was > recently asking if I thought there was value in adding a large-memory > system, say with 1TB of RAM. LKP is the system that generates these > kinds of automated bug reports and performance tests: > > http://lkml.org/lkml/2014/3/9/201 > > My gut reaction was that we'd probably be better served by putting > resources in to systems with higher core counts rather than lots of RAM. > I have encountered the occasional boot bug on my 1TB system, but it's > far from a frequent occurrence, and even more infrequent to encounter > things at runtime. > > Would folks agree with that? What kinds of tests, benchmarks, stress > tests, etc... do folks run that are both valuable and can only be run on > a system with a large amount of actual RAM? We had a sched-numa + kvm fail on really large systems the other day, but yeah in general such problems tend to be rare. Then again, without test coverage they will always be rare, for even if there were problems, nobody would notice :-) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757338AbaCRR0q (ORCPT ); Tue, 18 Mar 2014 13:26:46 -0400 Received: from qmta12.emeryville.ca.mail.comcast.net ([76.96.27.227]:49598 "EHLO qmta12.emeryville.ca.mail.comcast.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757151AbaCRR0m (ORCPT ); Tue, 18 Mar 2014 13:26:42 -0400 Date: Tue, 18 Mar 2014 12:26:40 -0500 (CDT) From: Christoph Lameter X-X-Sender: cl@nuc To: Peter Zijlstra cc: Dave Hansen , lsf@lists.linux-foundation.org, Linux-MM , Wu Fengguang , LKML Subject: Re: [Lsf] [LSF/MM TOPIC] Testing Large-Memory Hardware In-Reply-To: <20140318165059.GI22095@laptop.programming.kicks-ass.net> Message-ID: References: <5328753B.2050107@intel.com> <20140318165059.GI22095@laptop.programming.kicks-ass.net> Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 18 Mar 2014, Peter Zijlstra wrote: > > My gut reaction was that we'd probably be better served by putting > > resources in to systems with higher core counts rather than lots of RAM. > > I have encountered the occasional boot bug on my 1TB system, but it's > > far from a frequent occurrence, and even more infrequent to encounter > > things at runtime. > > > > Would folks agree with that? What kinds of tests, benchmarks, stress > > tests, etc... do folks run that are both valuable and can only be run on > > a system with a large amount of actual RAM? > > We had a sched-numa + kvm fail on really large systems the other day, > but yeah in general such problems tend to be rare. Then again, without > test coverage they will always be rare, for even if there were problems, > nobody would notice :-) SGI had systems out there up to few PB of RAM. There were a couple of tricks to get this going. Bootup time was pretty long. I/O has to be done carefully. The MM subsystem used to work with these sizes (I have not had a chance to verify that recently). This was Itanium with 64K page size so you had a factor of 16 less page structs to process. What I saw there is one of the reasons why I would like to see larger page support in the kernel. Managing massive amounts of 4k pages is creation far too much overhead.