From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Alan D. Brunelle" Date: Tue, 31 Aug 2010 11:32:07 +0000 Subject: Re: blktrace could cause system hang under 1G memory size and 20 Message-Id: <4C7CE837.2020401@hp.com> List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-btrace@vger.kernel.org On 08/31/2010 02:32 AM, Xuekun Hu wrote: > Hi, Guys > > My Linux system (2.6.18, x86_64, RHEL5) only has 1G memory, and I > would like to use blktrace (1.0.1) to capture 20 minutes of IO traces > with medium IO traffics. After 20 minutes, system would hang. I think > at this phase, blktrace is exiting and printing the summary section > that number of events captured on each cpu. > > I'm guessing the system hang due to out of memory. (I can ping the > system, but can't login). I'm wondering is there a workaround? > > Thx, Xuekun > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrace" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Well, you'd have to prove that the system was out of memory - note that blktrace statically allocates it's memory at the beginning of the run (or at least, used to). So it's direct use of memory shouldn't expand over time. Indirectly - as it puts data out to disk - it could be consuming buffers for that operation. One workaround (to try) would be to go into network mode and send the data to some other machine to process. Of course, network traffic consumes its own buffers as well... Another option is to limit the traces to collect - if there are only certain things you are interested in, you could limit collection & saving to just those traces. [One thing to disable would be the I/O scheduler traffic if you are not interested in that - I found in the past that some of those (CFQ) could generate a tremendous amount of traces.] Regards, Alan