* makefumpfile tool to estimate vmcore file size @ 2013-07-17 7:58 Baoquan 2013-07-24 7:06 ` Atsushi Kumagai 0 siblings, 1 reply; 4+ messages in thread From: Baoquan @ 2013-07-17 7:58 UTC (permalink / raw) To: Atsushi Kumagai; +Cc: Kexec-ml Hi Atsushi, Our customer want us to provide a tool to estimate the required dump file size based on the current system memory footprint. The following is detailed requirement I tried to conclude, what's your opinion? In customer's place there are thousands of machines and they don't want to budget for significant increases in storage if unnecessary. This becomes particularly expensive with large memory (1tb+) systems booting off san disk. Customer would like to achieve this by below example: ########################################################## #makedumpfile -d31 -c/-l/-p TYPE PAGES INCLUDED Zero Page x no Cache Without Private x no Cache With Private x no User Data x no Free Page x no Kernel Code x yes Kernel Data x yes Total Pages on system: 311000 (Just for example) Total Pages included in kdump: 160000 (Just for example) Estimated vmcore file size: 48000 (30% compression ratio) ########################################################## By configured dump level, total pages included in kdump can be computed. Then with option which specify a compression algorithm, an estimated vmcore file size can be given. Though the estimated value is changed dynamically with time, it does give user a valuable reference. Thanks a lot Baoquan _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: makefumpfile tool to estimate vmcore file size 2013-07-17 7:58 makefumpfile tool to estimate vmcore file size Baoquan @ 2013-07-24 7:06 ` Atsushi Kumagai 2013-08-02 3:29 ` Baoquan He 0 siblings, 1 reply; 4+ messages in thread From: Atsushi Kumagai @ 2013-07-24 7:06 UTC (permalink / raw) To: bhe; +Cc: kexec Hello Baoquan, On Wed, 17 Jul 2013 15:58:30 +0800 Baoquan <bhe@redhat.com> wrote: > > Hi Atsushi, > > Our customer want us to provide a tool to estimate the required dump > file size based on the current system memory footprint. The following is > detailed requirement I tried to conclude, what's your opinion? > > In customer's place there are thousands of machines and they don't want > to budget for significant increases in storage if unnecessary. This > becomes particularly expensive with large memory (1tb+) systems booting > off san disk. > > Customer would like to achieve this by below example: > ########################################################## > #makedumpfile -d31 -c/-l/-p > > TYPE PAGES INCLUDED > > Zero Page x no > Cache Without Private x no > Cache With Private x no > User Data x no > Free Page x no > Kernel Code x yes > Kernel Data x yes > > Total Pages on system: 311000 (Just for example) > Total Pages included in kdump: 160000 (Just for example) > Estimated vmcore file size: 48000 (30% compression ratio) > ########################################################## Does this image mean that you want to run makedumpfile in 1st kernel without generating a actual dumpfile ? Unfortunately, makedumpfile can't work in 1st kernel because it only supports /proc/vmcore as input data. If you don't persist in doing in 1st kernel, your desire can be achieved by modifying print_report() and discarding a output data to /dev/null, and running makedumpfile via kdump as usual. > By configured dump level, total pages included in kdump can be computed. > Then with option which specify a compression algorithm, an estimated > vmcore file size can be given. Though the estimated value is changed > dynamically with time, it does give user a valuable reference. Compression ratio is very dependent on the memory usage. So I think it's difficult to estimate the size when any compression algorithm is specified. Thanks Atsushi Kumagai _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: makefumpfile tool to estimate vmcore file size 2013-07-24 7:06 ` Atsushi Kumagai @ 2013-08-02 3:29 ` Baoquan He 2013-08-23 9:26 ` Atsushi Kumagai 0 siblings, 1 reply; 4+ messages in thread From: Baoquan He @ 2013-08-02 3:29 UTC (permalink / raw) To: Atsushi Kumagai; +Cc: kexec [-- Attachment #1: Type: text/plain, Size: 3188 bytes --] Hi Atsushi, Sorry to reply so late. By discussing with customer, their idea is below: ******************************* The idea about a tool like this is that it works in the 1st kernel and that it will tell you how big the vmcore will be based on what filtering level and/or compression selected in kdump.conf. Our customer wrote a perl script to demonstrate this, I have attached it as attachment. This perl script only looks at uncompressed and dumplevel 31 - in the 1st kernel. Being told about the dump size when in the 2nd (kdump/kexec kernel) is somewhat pointless as you are not then in a position to adjust the destination filesystem to accommodate the vmcore size should you need to. This is proactively looking at what size is required for a vmcore. If the kdump.conf says "filter level 11, compress", a tool to estimate the vmcore size should take that into account, gauge what pages would be included in that, and roughly what size that would equate to compressed. The problem is that this is quite dynamic, so the perl script deliberately only looks at filter level 31 (trying to work the rest out was too hard for me to do). ********************************** Per discussion with customer, I think the tool is expected to work in 1st kernel. With configuration file which specify the fileter level and compression algorithm, a rough vmcore size can be given. Since they maybe have thousands of servers, an estimated vmcore size can be very helpful to refer to. Baoquan Thanks a lot On 07/24/13 at 04:06pm, Atsushi Kumagai wrote: > On Wed, 17 Jul 2013 15:58:30 +0800 > Baoquan <bhe@redhat.com> wrote: > > > #makedumpfile -d31 -c/-l/-p > > > > TYPE PAGES INCLUDED > > > > Zero Page x no > > Cache Without Private x no > > Cache With Private x no > > User Data x no > > Free Page x no > > Kernel Code x yes > > Kernel Data x yes > > > > Total Pages on system: 311000 (Just for example) > > Total Pages included in kdump: 160000 (Just for example) > > Estimated vmcore file size: 48000 (30% compression ratio) > > ########################################################## > > Does this image mean that you want to run makedumpfile in 1st > kernel without generating a actual dumpfile ? > Unfortunately, makedumpfile can't work in 1st kernel because it > only supports /proc/vmcore as input data. > > If you don't persist in doing in 1st kernel, your desire can be > achieved by modifying print_report() and discarding a output > data to /dev/null, and running makedumpfile via kdump as usual. > > > By configured dump level, total pages included in kdump can be computed. > > Then with option which specify a compression algorithm, an estimated > > vmcore file size can be given. Though the estimated value is changed > > dynamically with time, it does give user a valuable reference. > > Compression ratio is very dependent on the memory usage. > So I think it's difficult to estimate the size when any compression > algorithm is specified. > > > Thanks > Atsushi Kumagai > > _______________________________________________ > kexec mailing list > kexec@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/kexec [-- Attachment #2: vmcore-size-guess.pl --] [-- Type: application/x-perl, Size: 2021 bytes --] [-- Attachment #3: Type: text/plain, Size: 143 bytes --] _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: makefumpfile tool to estimate vmcore file size 2013-08-02 3:29 ` Baoquan He @ 2013-08-23 9:26 ` Atsushi Kumagai 0 siblings, 0 replies; 4+ messages in thread From: Atsushi Kumagai @ 2013-08-23 9:26 UTC (permalink / raw) To: bhe; +Cc: kexec Hello Baoquan, (2013/08/02 12:29), Baoquan He wrote: > Hi Atsushi, > > Sorry to reply so late. > By discussing with customer, their idea is below: > > ******************************* > The idea about a tool like this is that it works in the 1st kernel and > that it will tell you how big the vmcore will be based on what filtering > level and/or compression selected in kdump.conf. Our customer wrote a perl > script to demonstrate this, I have attached it as attachment. This perl > script only looks at uncompressed and dumplevel 31 - in the 1st kernel. > Being told about the dump size when in the 2nd (kdump/kexec kernel) is > somewhat pointless as you are not then in a position to adjust the > destination filesystem to accommodate the vmcore size should you need to. > This is proactively looking at what size is required for a vmcore. > > If the kdump.conf says "filter level 11, compress", a tool to estimate > the vmcore size should take that into account, gauge what pages would > be included in that, and roughly what size that would equate to > compressed. The problem is that this is quite dynamic, so the perl > script deliberately only looks at filter level 31 (trying to work the > rest out was too hard for me to do). > ********************************** > > Per discussion with customer, I think the tool is expected to work in > 1st kernel. With configuration file which specify the fileter level and > compression algorithm, a rough vmcore size can be given. Since they > maybe have thousands of servers, an estimated vmcore size can be very > helpful to refer to. I understand your customer's idea, I also think it's useful for such situation. I'm afraid that I can't take time for developing new feature now, but I'll accept the estimation feature if anyone makes that. IMHO, I think /proc/kcore is better interface for analyzing live system while it's ELF format. This is because that the logic for /proc/vmcore might can be reused with small fix. Thanks Atsushi Kumagai. > > Baoquan > Thanks a lot > > On 07/24/13 at 04:06pm, Atsushi Kumagai wrote: >> On Wed, 17 Jul 2013 15:58:30 +0800 >> Baoquan <bhe@redhat.com> wrote: >> >>> #makedumpfile -d31 -c/-l/-p >>> >>> TYPE PAGES INCLUDED >>> >>> Zero Page x no >>> Cache Without Private x no >>> Cache With Private x no >>> User Data x no >>> Free Page x no >>> Kernel Code x yes >>> Kernel Data x yes >>> >>> Total Pages on system: 311000 (Just for example) >>> Total Pages included in kdump: 160000 (Just for example) >>> Estimated vmcore file size: 48000 (30% compression ratio) >>> ########################################################## >> >> Does this image mean that you want to run makedumpfile in 1st >> kernel without generating a actual dumpfile ? >> Unfortunately, makedumpfile can't work in 1st kernel because it >> only supports /proc/vmcore as input data. >> >> If you don't persist in doing in 1st kernel, your desire can be >> achieved by modifying print_report() and discarding a output >> data to /dev/null, and running makedumpfile via kdump as usual. >> >>> By configured dump level, total pages included in kdump can be computed. >>> Then with option which specify a compression algorithm, an estimated >>> vmcore file size can be given. Though the estimated value is changed >>> dynamically with time, it does give user a valuable reference. >> >> Compression ratio is very dependent on the memory usage. >> So I think it's difficult to estimate the size when any compression >> algorithm is specified. >> >> >> Thanks >> Atsushi Kumagai >> >> _______________________________________________ >> kexec mailing list >> kexec@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/kexec _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-08-23 9:34 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-07-17 7:58 makefumpfile tool to estimate vmcore file size Baoquan 2013-07-24 7:06 ` Atsushi Kumagai 2013-08-02 3:29 ` Baoquan He 2013-08-23 9:26 ` Atsushi Kumagai
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox