* [PATCH V5 1/5] Add support for splitblock
2014-11-06 8:49 [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
@ 2014-11-06 8:49 ` Zhou Wenjian
2014-11-06 8:49 ` [PATCH V5 2/5] Add tools for reading and writing from splitblock table Zhou Wenjian
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Zhou Wenjian @ 2014-11-06 8:49 UTC (permalink / raw)
To: kexec
When --split option is specified, fair I/O workloads shoud be assigned
for each process. So the start and end pfn of each dumpfile should be
calculated with excluding unnecessary pages. However, it costs a lot of
time to execute excluding for the whole memory. That is why struct
SplitBlock exists. Struct SplitBlock is designed to manage memory, mainly
for recording the number of dumpable pages. We can use the number of
dumpable pages to calculate start and end pfn instead of execute excluding
for the whole memory.
The char array *table in struct SplitBlock is used to record the number of
dumpable pages.
The table entry size is calculated as
divideup(log2(splitblock_size / page_size), 8) bytes
The table entry size is calculated, so that the
space table taken will be small enough. And the code will also have a
good performence when the number of pages in one splitblock is big enough.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 31 +++++++++++++++++++++++++++++++
makedumpfile.h | 14 ++++++++++++++
2 files changed, 45 insertions(+), 0 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index b4d43d8..7017943 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -34,6 +34,7 @@ struct srcfile_table srcfile_table;
struct vm_table vt = { 0 };
struct DumpInfo *info = NULL;
+struct SplitBlock *splitblock = NULL;
char filename_stdout[] = FILENAME_STDOUT;
@@ -5685,6 +5686,36 @@ out:
return ret;
}
+/*
+ * cyclic_split mode:
+ * manage memory by splitblocks,
+ * divide memory into splitblocks
+ * use splitblock_table to record numbers of dumpable pages in each
+ * splitblock
+ */
+
+/*
+ * calculate entry size based on the amount of pages in one splitblock
+ */
+int
+calculate_entry_size(void)
+{
+ int entry_num = 1;
+ int count = 1;
+ int entry_size;
+
+ while (entry_num < splitblock->page_per_splitblock){
+ entry_num = entry_num << 1;
+ count++;
+ }
+
+ entry_size = count / BITPERBYTE;
+ if (count % BITPERBYTE)
+ entry_size++;
+
+ return entry_size;
+}
+
mdf_pfn_t
get_num_dumpable(void)
{
diff --git a/makedumpfile.h b/makedumpfile.h
index 96830b0..7d9c2e6 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1168,10 +1168,24 @@ struct DumpInfo {
*/
int (*page_is_buddy)(unsigned long flags, unsigned int _mapcount,
unsigned long private, unsigned int _count);
+ /*
+ * for cyclic_splitting mode, setup splitblock_size
+ */
+ long long splitblock_size;
};
extern struct DumpInfo *info;
/*
+ * for cyclic_splitting mode,Manage memory by splitblock
+ */
+struct SplitBlock{
+ char *table;
+ long long num;
+ long long page_per_splitblock;
+ int entry_size; /* counted by byte */
+};
+
+/*
* kernel VM-related data
*/
struct vm_table {
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH V5 2/5] Add tools for reading and writing from splitblock table
2014-11-06 8:49 [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
2014-11-06 8:49 ` [PATCH V5 1/5] Add support for splitblock Zhou Wenjian
@ 2014-11-06 8:49 ` Zhou Wenjian
2014-11-06 8:49 ` [PATCH V5 3/5] Add module of generating table Zhou Wenjian
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Zhou Wenjian @ 2014-11-06 8:49 UTC (permalink / raw)
To: kexec
The function added in this patch, is used for writing and reading value
from the char array in struct SplitBlock.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 29 +++++++++++++++++++++++++++++
1 files changed, 29 insertions(+), 0 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index 7017943..f86dfd2 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -5716,6 +5716,35 @@ calculate_entry_size(void)
return entry_size;
}
+void
+write_into_splitblock_table(char *entry,
+ unsigned long long value)
+{
+ char temp;
+ int i = 0;
+
+ while (i++ < splitblock->entry_size) {
+ temp = value & 0xff;
+ value = value >> BITPERBYTE;
+ *entry = temp;
+ entry++;
+ }
+}
+
+unsigned long long
+read_from_splitblock_table(char *entry)
+{
+ unsigned long long value = 0;
+ int i;
+
+ for (i = splitblock->entry_size; i > 0; i--) {
+ value = value << BITPERBYTE;
+ value += *(entry + i - 1) & 0xff;
+ }
+
+ return value;
+}
+
mdf_pfn_t
get_num_dumpable(void)
{
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH V5 3/5] Add module of generating table
2014-11-06 8:49 [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
2014-11-06 8:49 ` [PATCH V5 1/5] Add support for splitblock Zhou Wenjian
2014-11-06 8:49 ` [PATCH V5 2/5] Add tools for reading and writing from splitblock table Zhou Wenjian
@ 2014-11-06 8:49 ` Zhou Wenjian
2014-11-06 8:49 ` [PATCH V5 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile Zhou Wenjian
` (2 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Zhou Wenjian @ 2014-11-06 8:49 UTC (permalink / raw)
To: kexec
set block size and generate basic information of block table
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
makedumpfile.h | 4 ++
2 files changed, 111 insertions(+), 1 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index f86dfd2..a01c616 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -5208,7 +5208,14 @@ create_dump_bitmap(void)
if (info->flag_cyclic) {
if (!prepare_bitmap2_buffer_cyclic())
goto out;
- info->num_dumpable = get_num_dumpable_cyclic();
+ if (info->flag_split) {
+ if(!prepare_splitblock_table())
+ goto out;
+
+ info->num_dumpable = get_num_dumpable_cyclic_withsplit();
+ }else {
+ info->num_dumpable = get_num_dumpable_cyclic();
+ }
if (!info->flag_elf_dumpfile)
free_bitmap2_buffer_cyclic();
@@ -5745,6 +5752,60 @@ read_from_splitblock_table(char *entry)
return value;
}
+/*
+ * The splitblock size is specified as Kbyte with --splitblock-size <size> option.
+ * If not specified, set default value.
+ */
+int
+check_splitblock_size(void)
+{
+ if (info->splitblock_size) {
+ info->splitblock_size <<= 10;
+ if (info->splitblock_size == 0) {
+ ERRMSG("The splitblock size could not be 0. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+ if (info->splitblock_size % info->page_size != 0) {
+ ERRMSG("The splitblock size must be align to page_size. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+ } else {
+ info->splitblock_size = DEFAULT_SPLITBLOCK_SIZE;
+ }
+
+ return TRUE;
+}
+
+int
+prepare_splitblock_table(void)
+{
+ size_t table_size;
+
+ if(!check_splitblock_size())
+ return FALSE;
+ if ((splitblock = calloc(1, sizeof(struct SplitBlock))) == NULL) {
+ ERRMSG("Can't allocate memory for the splitblock. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+
+ splitblock->page_per_splitblock = info->splitblock_size / info->page_size;
+ splitblock->num = divideup(info->max_mapnr, splitblock->page_per_splitblock);
+ splitblock->entry_size = calculate_entry_size();
+ table_size = splitblock->entry_size * splitblock->num;
+
+ splitblock->table = (char *)calloc(sizeof(char), table_size);
+ if (!splitblock->table) {
+ ERRMSG("Can't allocate memory for the splitblock_table. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+
+ return TRUE;
+}
+
mdf_pfn_t
get_num_dumpable(void)
{
@@ -5760,6 +5821,45 @@ get_num_dumpable(void)
return num_dumpable;
}
+/*
+ * generate splitblock_table
+ * modified from function get_num_dumpable_cyclic
+ */
+mdf_pfn_t
+get_num_dumpable_cyclic_withsplit(void)
+{
+ mdf_pfn_t pfn, num_dumpable = 0;
+ mdf_pfn_t dumpable_pfn_num = 0, pfn_num = 0;
+ struct cycle cycle = {0};
+ int pos = 0;
+
+ pfn_memhole = info->max_mapnr;
+
+ for_each_cycle(0, info->max_mapnr, &cycle) {
+ if (!exclude_unnecessary_pages_cyclic(&cycle))
+ return FALSE;
+
+ if (info->flag_mem_usage)
+ exclude_zero_pages_cyclic(&cycle);
+
+ for (pfn = cycle.start_pfn; pfn < cycle.end_pfn; pfn++) {
+ if (is_dumpable_cyclic(info->partial_bitmap2, pfn, &cycle)) {
+ num_dumpable++;
+ dumpable_pfn_num++;
+ }
+ if (++pfn_num >= splitblock->page_per_splitblock) {
+ write_into_splitblock_table(splitblock->table + pos,
+ dumpable_pfn_num);
+ pos += splitblock->entry_size;
+ pfn_num = 0;
+ dumpable_pfn_num = 0;
+ }
+ }
+ }
+
+ return num_dumpable;
+}
+
mdf_pfn_t
get_num_dumpable_cyclic(void)
{
@@ -9717,6 +9817,12 @@ out:
if (info->page_buf != NULL)
free(info->page_buf);
free(info);
+
+ if (splitblock) {
+ if (splitblock->table)
+ free(splitblock->table);
+ free(splitblock);
+ }
}
free_elf_info();
diff --git a/makedumpfile.h b/makedumpfile.h
index 7d9c2e6..39c49b8 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1178,6 +1178,8 @@ extern struct DumpInfo *info;
/*
* for cyclic_splitting mode,Manage memory by splitblock
*/
+#define DEFAULT_SPLITBLOCK_SIZE (1LL << 30)
+
struct SplitBlock{
char *table;
long long num;
@@ -1888,9 +1890,11 @@ struct elf_prstatus {
* Function Prototype.
*/
mdf_pfn_t get_num_dumpable_cyclic(void);
+mdf_pfn_t get_num_dumpable_cyclic_withsplit(void);
int get_loads_dumpfile_cyclic(void);
int initial_xen(void);
unsigned long long get_free_memory_size(void);
int calculate_cyclic_buffer_size(void);
+int prepare_splitblock_table(void);
#endif /* MAKEDUMPFILE_H */
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH V5 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile
2014-11-06 8:49 [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
` (2 preceding siblings ...)
2014-11-06 8:49 ` [PATCH V5 3/5] Add module of generating table Zhou Wenjian
@ 2014-11-06 8:49 ` Zhou Wenjian
2014-11-24 7:18 ` Baoquan He
2014-11-06 8:49 ` [PATCH V5 5/5] Add support for --splitblock-size Zhou Wenjian
2014-11-07 2:00 ` [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Atsushi Kumagai
5 siblings, 1 reply; 10+ messages in thread
From: Zhou Wenjian @ 2014-11-06 8:49 UTC (permalink / raw)
To: kexec
When --split is specified in cyclic mode, start_pfn and end_pfn of each dumpfile
will be calculated to make each dumpfile have the same size.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++----
1 files changed, 68 insertions(+), 5 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index a01c616..aa19b0d 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -8217,6 +8217,65 @@ out:
return ret;
}
+/*
+ * calculate end_pfn of one dumpfile.
+ * try to make every output file have the same size.
+ * splitblock_table is used to reduce calculate time.
+ */
+
+#define CURRENT_SPLITBLOCK_PFN_NUM (*cur_splitblock_num * splitblock->page_per_splitblock)
+mdf_pfn_t
+calculate_end_pfn_by_splitblock(mdf_pfn_t start_pfn,
+ int *cur_splitblock_num)
+{
+ if (start_pfn >= info->max_mapnr)
+ return info->max_mapnr;
+
+ mdf_pfn_t end_pfn;
+ long long pfn_needed, offset;
+ char *splitblock_value_offset;
+
+ pfn_needed = info->num_dumpable / info->num_dumpfile;
+ offset = *cur_splitblock_num * splitblock->entry_size;
+ splitblock_value_offset = splitblock->table + offset;
+ end_pfn = start_pfn;
+
+ while (*cur_splitblock_num < splitblock->num && pfn_needed > 0) {
+ pfn_needed -= read_from_splitblock_table(splitblock_value_offset);
+ splitblock_value_offset += splitblock->entry_size;
+ ++*cur_splitblock_num;
+ }
+
+ end_pfn = CURRENT_SPLITBLOCK_PFN_NUM;
+ if (end_pfn > info->max_mapnr)
+ end_pfn = info->max_mapnr;
+
+ return end_pfn;
+}
+
+/*
+ * calculate start_pfn and end_pfn in each output file.
+ */
+static int setup_splitting_cyclic(void)
+{
+ int i;
+ mdf_pfn_t start_pfn, end_pfn;
+ int cur_splitblock_num = 0;
+ start_pfn = end_pfn = 0;
+
+ for (i = 0; i < info->num_dumpfile - 1; i++) {
+ start_pfn = end_pfn;
+ end_pfn = calculate_end_pfn_by_splitblock(start_pfn,
+ &cur_splitblock_num);
+ SPLITTING_START_PFN(i) = start_pfn;
+ SPLITTING_END_PFN(i) = end_pfn;
+ }
+ SPLITTING_START_PFN(info->num_dumpfile - 1) = end_pfn;
+ SPLITTING_END_PFN(info->num_dumpfile - 1) = info->max_mapnr;
+
+ return TRUE;
+}
+
int
setup_splitting(void)
{
@@ -8230,12 +8289,16 @@ setup_splitting(void)
return FALSE;
if (info->flag_cyclic) {
- for (i = 0; i < info->num_dumpfile; i++) {
- SPLITTING_START_PFN(i) = divideup(info->max_mapnr, info->num_dumpfile) * i;
- SPLITTING_END_PFN(i) = divideup(info->max_mapnr, info->num_dumpfile) * (i + 1);
+ int ret = FALSE;
+
+ if(!prepare_bitmap2_buffer_cyclic()){
+ free_bitmap_buffer();
+ return ret;
}
- if (SPLITTING_END_PFN(i-1) > info->max_mapnr)
- SPLITTING_END_PFN(i-1) = info->max_mapnr;
+ ret = setup_splitting_cyclic();
+ free_bitmap2_buffer_cyclic();
+
+ return ret;
} else {
initialize_2nd_bitmap(&bitmap2);
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH V5 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile
2014-11-06 8:49 ` [PATCH V5 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile Zhou Wenjian
@ 2014-11-24 7:18 ` Baoquan He
2014-11-24 7:28 ` "Zhou, Wenjian/周文剑"
0 siblings, 1 reply; 10+ messages in thread
From: Baoquan He @ 2014-11-24 7:18 UTC (permalink / raw)
To: Zhou Wenjian; +Cc: kexec
On 11/06/14 at 04:49pm, Zhou Wenjian wrote:
> +#define CURRENT_SPLITBLOCK_PFN_NUM (*cur_splitblock_num * splitblock->page_per_splitblock)
> +mdf_pfn_t
> +calculate_end_pfn_by_splitblock(mdf_pfn_t start_pfn,
> + int *cur_splitblock_num)
> +{
> + if (start_pfn >= info->max_mapnr)
> + return info->max_mapnr;
> +
> + mdf_pfn_t end_pfn;
> + long long pfn_needed, offset;
> + char *splitblock_value_offset;
> +
> + pfn_needed = info->num_dumpable / info->num_dumpfile;
> + offset = *cur_splitblock_num * splitblock->entry_size;
> + splitblock_value_offset = splitblock->table + offset;
> + end_pfn = start_pfn;
> +
> + while (*cur_splitblock_num < splitblock->num && pfn_needed > 0) {
> + pfn_needed -= read_from_splitblock_table(splitblock_value_offset);
> + splitblock_value_offset += splitblock->entry_size;
> + ++*cur_splitblock_num;
> + }
Hi Wenjian,
If splitblock->page_per_splitblock is 1G, and the system ram is 4G. Now
I set 8 dumpfiles to store the splitted vmcore, say dump0...7. Then
dump0/1/2/3 will store 1G of data, dump4/5/6/7 will be empty. So is it
OK?
Can we do anything to avoid this case or give some note to let people not
be panikked by empty dump files?
Thanks
Baoquan
> +
> + end_pfn = CURRENT_SPLITBLOCK_PFN_NUM;
> + if (end_pfn > info->max_mapnr)
> + end_pfn = info->max_mapnr;
> +
> + return end_pfn;
> +}
> +
> +/*
> + * calculate start_pfn and end_pfn in each output file.
> + */
> +static int setup_splitting_cyclic(void)
> +{
> + int i;
> + mdf_pfn_t start_pfn, end_pfn;
> + int cur_splitblock_num = 0;
> + start_pfn = end_pfn = 0;
> +
> + for (i = 0; i < info->num_dumpfile - 1; i++) {
> + start_pfn = end_pfn;
> + end_pfn = calculate_end_pfn_by_splitblock(start_pfn,
> + &cur_splitblock_num);
> + SPLITTING_START_PFN(i) = start_pfn;
> + SPLITTING_END_PFN(i) = end_pfn;
> + }
> + SPLITTING_START_PFN(info->num_dumpfile - 1) = end_pfn;
> + SPLITTING_END_PFN(info->num_dumpfile - 1) = info->max_mapnr;
> +
> + return TRUE;
> +}
> +
> int
> setup_splitting(void)
> {
> @@ -8230,12 +8289,16 @@ setup_splitting(void)
> return FALSE;
>
> if (info->flag_cyclic) {
> - for (i = 0; i < info->num_dumpfile; i++) {
> - SPLITTING_START_PFN(i) = divideup(info->max_mapnr, info->num_dumpfile) * i;
> - SPLITTING_END_PFN(i) = divideup(info->max_mapnr, info->num_dumpfile) * (i + 1);
> + int ret = FALSE;
> +
> + if(!prepare_bitmap2_buffer_cyclic()){
> + free_bitmap_buffer();
> + return ret;
> }
> - if (SPLITTING_END_PFN(i-1) > info->max_mapnr)
> - SPLITTING_END_PFN(i-1) = info->max_mapnr;
> + ret = setup_splitting_cyclic();
> + free_bitmap2_buffer_cyclic();
> +
> + return ret;
> } else {
> initialize_2nd_bitmap(&bitmap2);
>
> --
> 1.7.1
>
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH V5 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile
2014-11-24 7:18 ` Baoquan He
@ 2014-11-24 7:28 ` "Zhou, Wenjian/周文剑"
2014-11-24 8:07 ` Baoquan He
0 siblings, 1 reply; 10+ messages in thread
From: "Zhou, Wenjian/周文剑" @ 2014-11-24 7:28 UTC (permalink / raw)
To: Baoquan He; +Cc: kexec
On 11/24/2014 03:18 PM, Baoquan He wrote:
> On 11/06/14 at 04:49pm, Zhou Wenjian wrote:
>
>> +#define CURRENT_SPLITBLOCK_PFN_NUM (*cur_splitblock_num * splitblock->page_per_splitblock)
>> +mdf_pfn_t
>> +calculate_end_pfn_by_splitblock(mdf_pfn_t start_pfn,
>> + int *cur_splitblock_num)
>> +{
>> + if (start_pfn>= info->max_mapnr)
>> + return info->max_mapnr;
>> +
>> + mdf_pfn_t end_pfn;
>> + long long pfn_needed, offset;
>> + char *splitblock_value_offset;
>> +
>> + pfn_needed = info->num_dumpable / info->num_dumpfile;
>> + offset = *cur_splitblock_num * splitblock->entry_size;
>> + splitblock_value_offset = splitblock->table + offset;
>> + end_pfn = start_pfn;
>> +
>> + while (*cur_splitblock_num< splitblock->num&& pfn_needed> 0) {
>> + pfn_needed -= read_from_splitblock_table(splitblock_value_offset);
>> + splitblock_value_offset += splitblock->entry_size;
>> + ++*cur_splitblock_num;
>> + }
>
> Hi Wenjian,
>
> If splitblock->page_per_splitblock is 1G, and the system ram is 4G. Now
> I set 8 dumpfiles to store the splitted vmcore, say dump0...7. Then
> dump0/1/2/3 will store 1G of data, dump4/5/6/7 will be empty. So is it
> OK?
>
> Can we do anything to avoid this case or give some note to let people not
> be panikked by empty dump files?
>
> Thanks
> Baoquan
>
>
Hello Baoquan,
Actually, we have thought about it and talked at
http://lists.infradead.org/pipermail/kexec/2014-November/012880.html .
--
Thanks
Zhou Wenjian
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH V5 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile
2014-11-24 7:28 ` "Zhou, Wenjian/周文剑"
@ 2014-11-24 8:07 ` Baoquan He
0 siblings, 0 replies; 10+ messages in thread
From: Baoquan He @ 2014-11-24 8:07 UTC (permalink / raw)
To: "Zhou, Wenjian/周文剑"; +Cc: kexec
On 11/24/14 at 03:28pm, "Zhou, Wenjian/周文剑" wrote:
> On 11/24/2014 03:18 PM, Baoquan He wrote:
> >On 11/06/14 at 04:49pm, Zhou Wenjian wrote:
> >>+ while (*cur_splitblock_num< splitblock->num&& pfn_needed> 0) {
> >>+ pfn_needed -= read_from_splitblock_table(splitblock_value_offset);
> >>+ splitblock_value_offset += splitblock->entry_size;
> >>+ ++*cur_splitblock_num;
> >>+ }
> >
> >Hi Wenjian,
> >
> >If splitblock->page_per_splitblock is 1G, and the system ram is 4G. Now
> >I set 8 dumpfiles to store the splitted vmcore, say dump0...7. Then
> >dump0/1/2/3 will store 1G of data, dump4/5/6/7 will be empty. So is it
> >OK?
> >
> >Can we do anything to avoid this case or give some note to let people not
> >be panikked by empty dump files?
> >
> >Thanks
> >Baoquan
> >
> >
>
> Hello Baoquan,
>
> Actually, we have thought about it and talked at
> http://lists.infradead.org/pipermail/kexec/2014-November/012880.html .
OK, thanks for telling.
>
> --
> Thanks
> Zhou Wenjian
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH V5 5/5] Add support for --splitblock-size
2014-11-06 8:49 [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
` (3 preceding siblings ...)
2014-11-06 8:49 ` [PATCH V5 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile Zhou Wenjian
@ 2014-11-06 8:49 ` Zhou Wenjian
2014-11-07 2:00 ` [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Atsushi Kumagai
5 siblings, 0 replies; 10+ messages in thread
From: Zhou Wenjian @ 2014-11-06 8:49 UTC (permalink / raw)
To: kexec
Use --splitblock-size to specify splitblock size (KB)
When --split is specified in cyclic mode,splitblock table will be
generated in create_dump_bitmap().
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.8 | 10 ++++++++++
makedumpfile.c | 4 ++++
makedumpfile.h | 1 +
print_info.c | 5 +++++
4 files changed, 20 insertions(+), 0 deletions(-)
diff --git a/makedumpfile.8 b/makedumpfile.8
index 9cb12c0..5e121fd 100644
--- a/makedumpfile.8
+++ b/makedumpfile.8
@@ -386,6 +386,16 @@ size, so ordinary users don't need to specify this option.
# makedumpfile \-\-cyclic\-buffer 1024 \-d 31 \-x vmlinux /proc/vmcore dumpfile
.TP
+\fB\-\-splitblock\-size\fR \fIsplitblock_size\fR
+Specify the splitblock size in kilo bytes for analysis in the cyclic mode with --split.
+If --splitblock N is specified, difference of each splitted dumpfile size is at most N
+kilo bytes.
+.br
+.B Example:
+.br
+# makedumpfile \-\-splitblock\-size 1024 \-d 31 \-x vmlinux \-\-split /proc/vmcore dumpfile1 dumpfile2
+
+.TP
\fB\-\-non\-cyclic\fR
Running in the non-cyclic mode, this mode uses the old filtering logic same as v1.4.4 or before.
If you feel the cyclic mode is too slow, please try this mode.
diff --git a/makedumpfile.c b/makedumpfile.c
index aa19b0d..7a54680 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -9569,6 +9569,7 @@ static struct option longopts[] = {
{"eppic", required_argument, NULL, OPT_EPPIC},
{"non-mmap", no_argument, NULL, OPT_NON_MMAP},
{"mem-usage", no_argument, NULL, OPT_MEM_USAGE},
+ {"splitblock-size", required_argument, NULL, OPT_SPLITBLOCK_SIZE},
{0, 0, 0, 0}
};
@@ -9709,6 +9710,9 @@ main(int argc, char *argv[])
case OPT_CYCLIC_BUFFER:
info->bufsize_cyclic = atoi(optarg);
break;
+ case OPT_SPLITBLOCK_SIZE:
+ info->splitblock_size = atoi(optarg);
+ break;
case '?':
MSG("Commandline parameter is invalid.\n");
MSG("Try `makedumpfile --help' for more information.\n");
diff --git a/makedumpfile.h b/makedumpfile.h
index 39c49b8..d195069 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1885,6 +1885,7 @@ struct elf_prstatus {
#define OPT_EPPIC OPT_START+12
#define OPT_NON_MMAP OPT_START+13
#define OPT_MEM_USAGE OPT_START+14
+#define OPT_SPLITBLOCK_SIZE OPT_START+15
/*
* Function Prototype.
diff --git a/print_info.c b/print_info.c
index f6342d3..16830b2 100644
--- a/print_info.c
+++ b/print_info.c
@@ -203,6 +203,11 @@ print_usage(void)
MSG(" By default, BUFFER_SIZE will be calculated automatically depending on\n");
MSG(" system memory size, so ordinary users don't need to specify this option.\n");
MSG("\n");
+ MSG(" [--splitblock-size SPLITBLOCK_SIZE]:\n");
+ MSG(" Specify the splitblock size in kilo bytes for analysis in the cyclic mode\n");
+ MSG(" with --split.If --splitblock N is specified, difference of each splitted\n");
+ MSG(" dumpfile size is at most N kilo bytes.\n");
+ MSG("\n");
MSG(" [--non-cyclic]:\n");
MSG(" Running in the non-cyclic mode, this mode uses the old filtering logic\n");
MSG(" same as v1.4.4 or before.\n");
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 10+ messages in thread* RE: [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time
2014-11-06 8:49 [PATCH V5 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
` (4 preceding siblings ...)
2014-11-06 8:49 ` [PATCH V5 5/5] Add support for --splitblock-size Zhou Wenjian
@ 2014-11-07 2:00 ` Atsushi Kumagai
5 siblings, 0 replies; 10+ messages in thread
From: Atsushi Kumagai @ 2014-11-07 2:00 UTC (permalink / raw)
To: zhouwj-fnst@cn.fujitsu.com; +Cc: kexec@lists.infradead.org
Hello Zhou,
>v4->v5:
> 1.fix the bug that last end_pfn can be shorter than the max_mapnr
Looks good to me, thanks for all your work !
I'll merge this version into v1.5.8.
Thanks,
Atsushi Kumagai
>v3->v4:
> 1.address Atsushi Kumagai's comments about using divideup
>
>v2->v3:
> 1.remove the filtering for incomplete block (previous second pass) and adjust
> relevant codes
> 2.address HATAYAMA's comments about coding style
>
>v1->v2:
> 1.use splitblock instead of block
> 2.add restriction (align to the page size) to splitblock size
> 3.adjust the position of prepare_splitblock_table and check the return code
> 4.use --splitblock-size to specify splitblock size and modify the print_info.c
>
>the v2 test result is:
>
> cyc-buf 128 256 512 1024 2048 4096
>splblk-size
>2M 24.05(0.65) 24.04(0.65) 24.58(0.65) 24.15(0.64) 24.14(0.64) 49.05(25.46)
>4M 23.98(0.65) 24.01(0.65) 24.29(0.65) 24.20(0.64) 24.18(0.66) 49.04(25.46)
>8M 24.02(0.65) 24.03(0.65) 24.25(0.65) 24.26(0.70) 24.15(0.64) 48.98(25.44)
>16M 24.01(0.65) 24.01(0.65) 24.30(0.65) 24.19(0.64) 24.12(0.65) 48.99(25.45)
>32M 23.97(0.65) 24.06(0.73) 24.23(0.65) 24.17(0.64) 24.19(0.64) 48.97(25.50)
>64M 24.06(0.66) 24.07(0.66) 24.27(0.66) 24.16(0.65) 24.17(0.65) 48.98(25.49)
>128M 24.03(0.67) 24.00(0.67) 24.27(0.66) 24.22(0.66) 24.19(0.66) 48.98(25.48)
>256M 24.12(0.67) 23.99(0.67) 24.27(0.67) 24.17(0.66) 24.12(0.66) 49.04(25.49)
>512M 24.06(0.70) 24.08(0.70) 24.26(0.70) 24.14(0.71) 24.19(0.70) 49.13(25.64)
>1G 24.20(0.82) 24.13(0.81) 24.36(0.81) 24.31(0.80) 24.33(0.81) 49.28(25.75)
>2G 24.19(0.81) 24.22(0.81) 24.37(0.81) 24.29(0.80) 24.28(0.82) 49.30(25.78)
>4G 25.29(1.90) 25.26(1.91) 25.49(1.91) 25.41(1.89) 25.50(1.90) 49.99(26.45)
>8G 25.33(1.90) 26.60(3.23) 26.87(3.21) 26.71(3.23) 26.64(3.22) 51.27(27.73)
>16G 25.28(1.90) 26.52(3.21) 29.47(5.86) 29.34(5.84) 29.38(5.86) 53.99(30.40)
>
>
>the latest test result turns to:
>
> cyc-buf 128 256 512 1024 2048 4096
>splblk-size
>2M 23.34(0.00) 23.36(0.00) 23.51(0.00) 23.86(0.00) 23.48(0.00) 23.45(0.00)
>4M 23.36(0.00) 23.32(0.00) 23.62(0.00) 23.48(0.00) 23.53(0.00) 23.45(0.00)
>8M 23.33(0.00) 23.46(0.00) 23.61(0.00) 23.77(0.00) 23.52(0.00) 23.53(0.00)
>16M 23.33(0.00) 23.38(0.00) 23.57(0.00) 23.47(0.00) 23.55(0.00) 23.49(0.00)
>32M 23.40(0.00) 23.31(0.00) 23.64(0.00) 23.51(0.00) 23.50(0.00) 23.55(0.00)
>64M 23.39(0.00) 23.41(0.00) 23.62(0.00) 23.47(0.00) 23.46(0.00) 23.50(0.00)
>128M 23.36(0.00) 23.34(0.00) 23.59(0.00) 23.46(0.00) 23.46(0.00) 23.50(0.00)
>256M 23.32(0.00) 23.36(0.00) 23.57(0.00) 23.49(0.00) 23.45(0.00) 23.52(0.00)
>512M 23.32(0.00) 23.32(0.00) 23.63(0.00) 23.49(0.00) 23.52(0.00) 23.51(0.00)
>1G 23.38(0.00) 23.32(0.00) 23.60(0.00) 23.54(0.00) 23.49(0.00) 23.54(0.00)
>2G 23.34(0.00) 23.36(0.00) 23.62(0.00) 23.52(0.00) 23.52(0.00) 23.50(0.00)
>4G 23.36(0.00) 23.37(0.00) 23.60(0.00) 23.48(0.00) 23.56(0.00) 23.55(0.00)
>8G 23.37(0.00) 23.39(0.00) 23.52(0.00) 23.53(0.00) 23.46(0.00) 23.48(0.00)
>16G 23.39(0.00) 23.41(0.00) 23.85(0.00) 23.48(0.00) 23.50(0.00) 23.45(0.00)
>
>
>
>Zhou Wenjian (5):
> Add support for splitblock
> Add tools for reading and writing from splitblock table
> Add module of generating table
> Add module of calculating start_pfn and end_pfn in each dumpfile
> Add support for --splitblock-size
>
> makedumpfile.8 | 10 +++
> makedumpfile.c | 245 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
> makedumpfile.h | 19 +++++
> print_info.c | 5 +
> 4 files changed, 273 insertions(+), 6 deletions(-)
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 10+ messages in thread