* [PATCH V3 1/5] Add support for splitblock
2014-10-30 9:45 [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
@ 2014-10-30 9:45 ` Zhou Wenjian
2014-10-30 9:45 ` [PATCH V3 2/5] Add tools for reading and writing from splitblock table Zhou Wenjian
` (4 subsequent siblings)
5 siblings, 0 replies; 11+ messages in thread
From: Zhou Wenjian @ 2014-10-30 9:45 UTC (permalink / raw)
To: kexec
When --split option is specified, fair I/O workloads shoud be assigned
for each process. So the start and end pfn of each dumpfile should be
calculated with excluding unnecessary pages. However, it costs a lot of
time to execute excluding for the whole memory. That is why struct
SplitBlock exists. Struct SplitBlock is designed to manage memory, mainly
for recording the number of dumpable pages. We can use the number of
dumpable pages to calculate start and end pfn instead of execute excluding
for the whole memory.
The char array *table in struct SplitBlock is used to record the number of
dumpable pages.
The table entry size is calculated as
divideup(log2(splitblock_size / page_size), 8) bytes
The table entry size is calculated, so that the
space table taken will be small enough. And the code will also have a
good performence when the number of pages in one splitblock is big enough.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 31 +++++++++++++++++++++++++++++++
makedumpfile.h | 14 ++++++++++++++
2 files changed, 45 insertions(+), 0 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index b4d43d8..7017943 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -34,6 +34,7 @@ struct srcfile_table srcfile_table;
struct vm_table vt = { 0 };
struct DumpInfo *info = NULL;
+struct SplitBlock *splitblock = NULL;
char filename_stdout[] = FILENAME_STDOUT;
@@ -5685,6 +5686,36 @@ out:
return ret;
}
+/*
+ * cyclic_split mode:
+ * manage memory by splitblocks,
+ * divide memory into splitblocks
+ * use splitblock_table to record numbers of dumpable pages in each
+ * splitblock
+ */
+
+/*
+ * calculate entry size based on the amount of pages in one splitblock
+ */
+int
+calculate_entry_size(void)
+{
+ int entry_num = 1;
+ int count = 1;
+ int entry_size;
+
+ while (entry_num < splitblock->page_per_splitblock){
+ entry_num = entry_num << 1;
+ count++;
+ }
+
+ entry_size = count / BITPERBYTE;
+ if (count % BITPERBYTE)
+ entry_size++;
+
+ return entry_size;
+}
+
mdf_pfn_t
get_num_dumpable(void)
{
diff --git a/makedumpfile.h b/makedumpfile.h
index 96830b0..7d9c2e6 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1168,10 +1168,24 @@ struct DumpInfo {
*/
int (*page_is_buddy)(unsigned long flags, unsigned int _mapcount,
unsigned long private, unsigned int _count);
+ /*
+ * for cyclic_splitting mode, setup splitblock_size
+ */
+ long long splitblock_size;
};
extern struct DumpInfo *info;
/*
+ * for cyclic_splitting mode,Manage memory by splitblock
+ */
+struct SplitBlock{
+ char *table;
+ long long num;
+ long long page_per_splitblock;
+ int entry_size; /* counted by byte */
+};
+
+/*
* kernel VM-related data
*/
struct vm_table {
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH V3 2/5] Add tools for reading and writing from splitblock table
2014-10-30 9:45 [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
2014-10-30 9:45 ` [PATCH V3 1/5] Add support for splitblock Zhou Wenjian
@ 2014-10-30 9:45 ` Zhou Wenjian
2014-11-05 4:18 ` Atsushi Kumagai
2014-10-30 9:45 ` [PATCH V3 3/5] Add module of generating table Zhou Wenjian
` (3 subsequent siblings)
5 siblings, 1 reply; 11+ messages in thread
From: Zhou Wenjian @ 2014-10-30 9:45 UTC (permalink / raw)
To: kexec
The function added in this patch, is used for writing and reading value
from the char array in struct SplitBlock.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 29 +++++++++++++++++++++++++++++
1 files changed, 29 insertions(+), 0 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index 7017943..f86dfd2 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -5716,6 +5716,35 @@ calculate_entry_size(void)
return entry_size;
}
+void
+write_into_splitblock_table(char *entry,
+ unsigned long long value)
+{
+ char temp;
+ int i = 0;
+
+ while (i++ < splitblock->entry_size) {
+ temp = value & 0xff;
+ value = value >> BITPERBYTE;
+ *entry = temp;
+ entry++;
+ }
+}
+
+unsigned long long
+read_from_splitblock_table(char *entry)
+{
+ unsigned long long value = 0;
+ int i;
+
+ for (i = splitblock->entry_size; i > 0; i--) {
+ value = value << BITPERBYTE;
+ value += *(entry + i - 1) & 0xff;
+ }
+
+ return value;
+}
+
mdf_pfn_t
get_num_dumpable(void)
{
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 11+ messages in thread* RE: [PATCH V3 2/5] Add tools for reading and writing from splitblock table
2014-10-30 9:45 ` [PATCH V3 2/5] Add tools for reading and writing from splitblock table Zhou Wenjian
@ 2014-11-05 4:18 ` Atsushi Kumagai
2014-11-05 10:31 ` "Zhou, Wenjian/周文剑"
0 siblings, 1 reply; 11+ messages in thread
From: Atsushi Kumagai @ 2014-11-05 4:18 UTC (permalink / raw)
To: zhouwj-fnst@cn.fujitsu.com; +Cc: kexec@lists.infradead.org
>The function added in this patch, is used for writing and reading value
>from the char array in struct SplitBlock.
>
>Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
>Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
>---
> makedumpfile.c | 29 +++++++++++++++++++++++++++++
> 1 files changed, 29 insertions(+), 0 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index 7017943..f86dfd2 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -5716,6 +5716,35 @@ calculate_entry_size(void)
> return entry_size;
> }
>
>+void
>+write_into_splitblock_table(char *entry,
>+ unsigned long long value)
>+{
>+ char temp;
>+ int i = 0;
>+
>+ while (i++ < splitblock->entry_size) {
>+ temp = value & 0xff;
>+ value = value >> BITPERBYTE;
>+ *entry = temp;
>+ entry++;
>+ }
>+}
>+
>+unsigned long long
>+read_from_splitblock_table(char *entry)
>+{
>+ unsigned long long value = 0;
>+ int i;
>+
>+ for (i = splitblock->entry_size; i > 0; i--) {
>+ value = value << BITPERBYTE;
>+ value += *(entry + i - 1) & 0xff;
>+ }
Did you forget to reflect the HATAYAMA-san's comment ?
>> + for (i = splitblock->entry_size; i > 0; i--) {
>> + ret = ret << BITPERBYTE;
>> + ret += *(splitblock_inner + i - 1) & 0xff;
>
> The & 0xff is necessary? because splitblock_inner is of type char *.
Thanks,
Atsushi Kumagai
>+
>+ return value;
>+}
>+
> mdf_pfn_t
> get_num_dumpable(void)
> {
>--
>1.7.1
>
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH V3 2/5] Add tools for reading and writing from splitblock table
2014-11-05 4:18 ` Atsushi Kumagai
@ 2014-11-05 10:31 ` "Zhou, Wenjian/周文剑"
2014-11-06 4:20 ` Atsushi Kumagai
0 siblings, 1 reply; 11+ messages in thread
From: "Zhou, Wenjian/周文剑" @ 2014-11-05 10:31 UTC (permalink / raw)
To: Atsushi Kumagai; +Cc: kexec@lists.infradead.org
On 11/05/2014 12:18 PM, Atsushi Kumagai wrote:
>> +unsigned long long
>> >+read_from_splitblock_table(char *entry)
>> >+{
>> >+ unsigned long long value = 0;
>> >+ int i;
>> >+
>> >+ for (i = splitblock->entry_size; i> 0; i--) {
>> >+ value = value<< BITPERBYTE;
>> >+ value += *(entry + i - 1)& 0xff;
>> >+ }
> Did you forget to reflect the HATAYAMA-san's comment ?
>
I think it is needed.
For example:
if *(entry + i - 1) == 0xfa
without & 0xff:
it will turn to: value += 0xfffffffffffffffa
with & 0xff:
it will be: value += 0xfffffffffffffffa & 0xff
Thanks
Zhou Wenjian
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 11+ messages in thread* RE: [PATCH V3 2/5] Add tools for reading and writing from splitblock table
2014-11-05 10:31 ` "Zhou, Wenjian/周文剑"
@ 2014-11-06 4:20 ` Atsushi Kumagai
0 siblings, 0 replies; 11+ messages in thread
From: Atsushi Kumagai @ 2014-11-06 4:20 UTC (permalink / raw)
To: zhouwj-fnst@cn.fujitsu.com; +Cc: kexec@lists.infradead.org
>On 11/05/2014 12:18 PM, Atsushi Kumagai wrote:
>>> +unsigned long long
>>> >+read_from_splitblock_table(char *entry)
>>> >+{
>>> >+ unsigned long long value = 0;
>>> >+ int i;
>>> >+
>>> >+ for (i = splitblock->entry_size; i> 0; i--) {
>>> >+ value = value<< BITPERBYTE;
>>> >+ value += *(entry + i - 1)& 0xff;
>>> >+ }
>> Did you forget to reflect the HATAYAMA-san's comment ?
>>
>
>I think it is needed.
>For example:
> if *(entry + i - 1) == 0xfa
> without & 0xff:
> it will turn to: value += 0xfffffffffffffffa
> with & 0xff:
> it will be: value += 0xfffffffffffffffa & 0xff
Ah, I see, you meant "sign extension".
OK, I understand.
Thanks,
Atsushi Kumagai
>
>Thanks
>Zhou Wenjian
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH V3 3/5] Add module of generating table
2014-10-30 9:45 [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
2014-10-30 9:45 ` [PATCH V3 1/5] Add support for splitblock Zhou Wenjian
2014-10-30 9:45 ` [PATCH V3 2/5] Add tools for reading and writing from splitblock table Zhou Wenjian
@ 2014-10-30 9:45 ` Zhou Wenjian
2014-11-05 4:18 ` Atsushi Kumagai
2014-10-30 9:45 ` [PATCH V3 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile Zhou Wenjian
` (2 subsequent siblings)
5 siblings, 1 reply; 11+ messages in thread
From: Zhou Wenjian @ 2014-10-30 9:45 UTC (permalink / raw)
To: kexec
set block size and generate basic information of block table
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
makedumpfile.h | 4 ++
2 files changed, 112 insertions(+), 1 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index f86dfd2..9ead207 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -5208,7 +5208,14 @@ create_dump_bitmap(void)
if (info->flag_cyclic) {
if (!prepare_bitmap2_buffer_cyclic())
goto out;
- info->num_dumpable = get_num_dumpable_cyclic();
+ if (info->flag_split) {
+ if(!prepare_splitblock_table())
+ goto out;
+
+ info->num_dumpable = get_num_dumpable_cyclic_withsplit();
+ }else {
+ info->num_dumpable = get_num_dumpable_cyclic();
+ }
if (!info->flag_elf_dumpfile)
free_bitmap2_buffer_cyclic();
@@ -5745,6 +5752,61 @@ read_from_splitblock_table(char *entry)
return value;
}
+/*
+ * The splitblock size is specified as Kbyte with --splitblock-size <size> option.
+ * If not specified, set default value.
+ */
+int
+check_splitblock_size(void)
+{
+ if (info->splitblock_size) {
+ info->splitblock_size <<= 10;
+ if (info->splitblock_size == 0) {
+ ERRMSG("The splitblock size could not be 0. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+ if (info->splitblock_size % info->page_size != 0) {
+ ERRMSG("The splitblock size must be align to page_size. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+ } else {
+ info->splitblock_size = DEFAULT_SPLITBLOCK_SIZE;
+ }
+
+ return TRUE;
+}
+
+int
+prepare_splitblock_table(void)
+{
+ size_t table_size;
+
+ if(!check_splitblock_size())
+ return FALSE;
+ if ((splitblock = calloc(1, sizeof(struct SplitBlock))) == NULL) {
+ ERRMSG("Can't allocate memory for the splitblock. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+
+ splitblock->page_per_splitblock = info->splitblock_size / info->page_size;
+ splitblock->num = (info->max_mapnr + splitblock->page_per_splitblock -1) /
+ splitblock->page_per_splitblock;
+ splitblock->entry_size = calculate_entry_size();
+ table_size = splitblock->entry_size * splitblock->num;
+
+ splitblock->table = (char *)calloc(sizeof(char), table_size);
+ if (!splitblock->table) {
+ ERRMSG("Can't allocate memory for the splitblock_table. %s.\n",
+ strerror(errno));
+ return FALSE;
+ }
+
+ return TRUE;
+}
+
mdf_pfn_t
get_num_dumpable(void)
{
@@ -5760,6 +5822,45 @@ get_num_dumpable(void)
return num_dumpable;
}
+/*
+ * generate splitblock_table
+ * modified from function get_num_dumpable_cyclic
+ */
+mdf_pfn_t
+get_num_dumpable_cyclic_withsplit(void)
+{
+ mdf_pfn_t pfn, num_dumpable = 0;
+ mdf_pfn_t dumpable_pfn_num = 0, pfn_num = 0;
+ struct cycle cycle = {0};
+ int pos = 0;
+
+ pfn_memhole = info->max_mapnr;
+
+ for_each_cycle(0, info->max_mapnr, &cycle) {
+ if (!exclude_unnecessary_pages_cyclic(&cycle))
+ return FALSE;
+
+ if (info->flag_mem_usage)
+ exclude_zero_pages_cyclic(&cycle);
+
+ for (pfn = cycle.start_pfn; pfn < cycle.end_pfn; pfn++) {
+ if (is_dumpable_cyclic(info->partial_bitmap2, pfn, &cycle)) {
+ num_dumpable++;
+ dumpable_pfn_num++;
+ }
+ if (++pfn_num >= splitblock->page_per_splitblock) {
+ write_into_splitblock_table(splitblock->table + pos,
+ dumpable_pfn_num);
+ pos += splitblock->entry_size;
+ pfn_num = 0;
+ dumpable_pfn_num = 0;
+ }
+ }
+ }
+
+ return num_dumpable;
+}
+
mdf_pfn_t
get_num_dumpable_cyclic(void)
{
@@ -9717,6 +9818,12 @@ out:
if (info->page_buf != NULL)
free(info->page_buf);
free(info);
+
+ if (splitblock) {
+ if (splitblock->table)
+ free(splitblock->table);
+ free(splitblock);
+ }
}
free_elf_info();
diff --git a/makedumpfile.h b/makedumpfile.h
index 7d9c2e6..39c49b8 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1178,6 +1178,8 @@ extern struct DumpInfo *info;
/*
* for cyclic_splitting mode,Manage memory by splitblock
*/
+#define DEFAULT_SPLITBLOCK_SIZE (1LL << 30)
+
struct SplitBlock{
char *table;
long long num;
@@ -1888,9 +1890,11 @@ struct elf_prstatus {
* Function Prototype.
*/
mdf_pfn_t get_num_dumpable_cyclic(void);
+mdf_pfn_t get_num_dumpable_cyclic_withsplit(void);
int get_loads_dumpfile_cyclic(void);
int initial_xen(void);
unsigned long long get_free_memory_size(void);
int calculate_cyclic_buffer_size(void);
+int prepare_splitblock_table(void);
#endif /* MAKEDUMPFILE_H */
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 11+ messages in thread* RE: [PATCH V3 3/5] Add module of generating table
2014-10-30 9:45 ` [PATCH V3 3/5] Add module of generating table Zhou Wenjian
@ 2014-11-05 4:18 ` Atsushi Kumagai
0 siblings, 0 replies; 11+ messages in thread
From: Atsushi Kumagai @ 2014-11-05 4:18 UTC (permalink / raw)
To: zhouwj-fnst@cn.fujitsu.com; +Cc: kexec@lists.infradead.org
>set block size and generate basic information of block table
>
>Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
>Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
>---
> makedumpfile.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> makedumpfile.h | 4 ++
> 2 files changed, 112 insertions(+), 1 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index f86dfd2..9ead207 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -5208,7 +5208,14 @@ create_dump_bitmap(void)
> if (info->flag_cyclic) {
> if (!prepare_bitmap2_buffer_cyclic())
> goto out;
>- info->num_dumpable = get_num_dumpable_cyclic();
>+ if (info->flag_split) {
>+ if(!prepare_splitblock_table())
>+ goto out;
>+
>+ info->num_dumpable = get_num_dumpable_cyclic_withsplit();
>+ }else {
>+ info->num_dumpable = get_num_dumpable_cyclic();
>+ }
>
> if (!info->flag_elf_dumpfile)
> free_bitmap2_buffer_cyclic();
>@@ -5745,6 +5752,61 @@ read_from_splitblock_table(char *entry)
> return value;
> }
>
>+/*
>+ * The splitblock size is specified as Kbyte with --splitblock-size <size> option.
>+ * If not specified, set default value.
>+ */
>+int
>+check_splitblock_size(void)
>+{
>+ if (info->splitblock_size) {
>+ info->splitblock_size <<= 10;
>+ if (info->splitblock_size == 0) {
>+ ERRMSG("The splitblock size could not be 0. %s.\n",
>+ strerror(errno));
>+ return FALSE;
>+ }
>+ if (info->splitblock_size % info->page_size != 0) {
>+ ERRMSG("The splitblock size must be align to page_size. %s.\n",
>+ strerror(errno));
>+ return FALSE;
>+ }
>+ } else {
>+ info->splitblock_size = DEFAULT_SPLITBLOCK_SIZE;
>+ }
>+
>+ return TRUE;
>+}
>+
>+int
>+prepare_splitblock_table(void)
>+{
>+ size_t table_size;
>+
>+ if(!check_splitblock_size())
>+ return FALSE;
>+ if ((splitblock = calloc(1, sizeof(struct SplitBlock))) == NULL) {
>+ ERRMSG("Can't allocate memory for the splitblock. %s.\n",
>+ strerror(errno));
>+ return FALSE;
>+ }
>+
>+ splitblock->page_per_splitblock = info->splitblock_size / info->page_size;
>+ splitblock->num = (info->max_mapnr + splitblock->page_per_splitblock -1) /
>+ splitblock->page_per_splitblock;
Please use "divideup(info->max_mapnr, splitblock->page_per_splitblock)".
Thanks,
Atsushi Kumagai
>+ splitblock->entry_size = calculate_entry_size();
>+ table_size = splitblock->entry_size * splitblock->num;
>+
>+ splitblock->table = (char *)calloc(sizeof(char), table_size);
>+ if (!splitblock->table) {
>+ ERRMSG("Can't allocate memory for the splitblock_table. %s.\n",
>+ strerror(errno));
>+ return FALSE;
>+ }
>+
>+ return TRUE;
>+}
>+
> mdf_pfn_t
> get_num_dumpable(void)
> {
>@@ -5760,6 +5822,45 @@ get_num_dumpable(void)
> return num_dumpable;
> }
>
>+/*
>+ * generate splitblock_table
>+ * modified from function get_num_dumpable_cyclic
>+ */
>+mdf_pfn_t
>+get_num_dumpable_cyclic_withsplit(void)
>+{
>+ mdf_pfn_t pfn, num_dumpable = 0;
>+ mdf_pfn_t dumpable_pfn_num = 0, pfn_num = 0;
>+ struct cycle cycle = {0};
>+ int pos = 0;
>+
>+ pfn_memhole = info->max_mapnr;
>+
>+ for_each_cycle(0, info->max_mapnr, &cycle) {
>+ if (!exclude_unnecessary_pages_cyclic(&cycle))
>+ return FALSE;
>+
>+ if (info->flag_mem_usage)
>+ exclude_zero_pages_cyclic(&cycle);
>+
>+ for (pfn = cycle.start_pfn; pfn < cycle.end_pfn; pfn++) {
>+ if (is_dumpable_cyclic(info->partial_bitmap2, pfn, &cycle)) {
>+ num_dumpable++;
>+ dumpable_pfn_num++;
>+ }
>+ if (++pfn_num >= splitblock->page_per_splitblock) {
>+ write_into_splitblock_table(splitblock->table + pos,
>+ dumpable_pfn_num);
>+ pos += splitblock->entry_size;
>+ pfn_num = 0;
>+ dumpable_pfn_num = 0;
>+ }
>+ }
>+ }
>+
>+ return num_dumpable;
>+}
>+
> mdf_pfn_t
> get_num_dumpable_cyclic(void)
> {
>@@ -9717,6 +9818,12 @@ out:
> if (info->page_buf != NULL)
> free(info->page_buf);
> free(info);
>+
>+ if (splitblock) {
>+ if (splitblock->table)
>+ free(splitblock->table);
>+ free(splitblock);
>+ }
> }
> free_elf_info();
>
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 7d9c2e6..39c49b8 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -1178,6 +1178,8 @@ extern struct DumpInfo *info;
> /*
> * for cyclic_splitting mode,Manage memory by splitblock
> */
>+#define DEFAULT_SPLITBLOCK_SIZE (1LL << 30)
>+
> struct SplitBlock{
> char *table;
> long long num;
>@@ -1888,9 +1890,11 @@ struct elf_prstatus {
> * Function Prototype.
> */
> mdf_pfn_t get_num_dumpable_cyclic(void);
>+mdf_pfn_t get_num_dumpable_cyclic_withsplit(void);
> int get_loads_dumpfile_cyclic(void);
> int initial_xen(void);
> unsigned long long get_free_memory_size(void);
> int calculate_cyclic_buffer_size(void);
>+int prepare_splitblock_table(void);
>
> #endif /* MAKEDUMPFILE_H */
>--
>1.7.1
>
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH V3 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile
2014-10-30 9:45 [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
` (2 preceding siblings ...)
2014-10-30 9:45 ` [PATCH V3 3/5] Add module of generating table Zhou Wenjian
@ 2014-10-30 9:45 ` Zhou Wenjian
2014-10-30 9:45 ` [PATCH V3 5/5] Add support for --splitblock-size Zhou Wenjian
2014-11-04 1:51 ` [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time "Zhou, Wenjian/周文剑"
5 siblings, 0 replies; 11+ messages in thread
From: Zhou Wenjian @ 2014-10-30 9:45 UTC (permalink / raw)
To: kexec
When --split is specified in cyclic mode, start_pfn and end_pfn of each dumpfile
will be calculated to make each dumpfile have the same size.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++----
1 files changed, 66 insertions(+), 5 deletions(-)
diff --git a/makedumpfile.c b/makedumpfile.c
index 9ead207..1468566 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -8218,6 +8218,63 @@ out:
return ret;
}
+/*
+ * calculate end_pfn of one dumpfile.
+ * try to make every output file have the same size.
+ * splitblock_table is used to reduce calculate time.
+ */
+
+#define CURRENT_SPLITBLOCK_PFN_NUM (*cur_splitblock_num * splitblock->page_per_splitblock)
+mdf_pfn_t
+calculate_end_pfn_by_splitblock(mdf_pfn_t start_pfn,
+ int *cur_splitblock_num)
+{
+ if (start_pfn >= info->max_mapnr)
+ return info->max_mapnr;
+
+ mdf_pfn_t end_pfn;
+ long long pfn_needed, offset;
+ char *splitblock_value_offset;
+
+ pfn_needed = info->num_dumpable / info->num_dumpfile;
+ offset = *cur_splitblock_num * splitblock->entry_size;
+ splitblock_value_offset = splitblock->table + offset;
+ end_pfn = start_pfn;
+
+ while (*cur_splitblock_num < splitblock->num && pfn_needed > 0) {
+ pfn_needed -= read_from_splitblock_table(splitblock_value_offset);
+ splitblock_value_offset += splitblock->entry_size;
+ ++*cur_splitblock_num;
+ }
+
+ end_pfn = CURRENT_SPLITBLOCK_PFN_NUM;
+ if (end_pfn > info->max_mapnr)
+ end_pfn = info->max_mapnr;
+
+ return end_pfn;
+}
+
+/*
+ * calculate start_pfn and end_pfn in each output file.
+ */
+static int setup_splitting_cyclic(void)
+{
+ int i;
+ mdf_pfn_t start_pfn, end_pfn;
+ int cur_splitblock_num = 0;
+ start_pfn = end_pfn = 0;
+
+ for (i = 0; i < info->num_dumpfile; i++) {
+ start_pfn = end_pfn;
+ end_pfn = calculate_end_pfn_by_splitblock(start_pfn,
+ &cur_splitblock_num);
+ SPLITTING_START_PFN(i) = start_pfn;
+ SPLITTING_END_PFN(i) = end_pfn;
+ }
+
+ return TRUE;
+}
+
int
setup_splitting(void)
{
@@ -8231,12 +8288,16 @@ setup_splitting(void)
return FALSE;
if (info->flag_cyclic) {
- for (i = 0; i < info->num_dumpfile; i++) {
- SPLITTING_START_PFN(i) = divideup(info->max_mapnr, info->num_dumpfile) * i;
- SPLITTING_END_PFN(i) = divideup(info->max_mapnr, info->num_dumpfile) * (i + 1);
+ int ret = FALSE;
+
+ if(!prepare_bitmap2_buffer_cyclic()){
+ free_bitmap_buffer();
+ return ret;
}
- if (SPLITTING_END_PFN(i-1) > info->max_mapnr)
- SPLITTING_END_PFN(i-1) = info->max_mapnr;
+ ret = setup_splitting_cyclic();
+ free_bitmap2_buffer_cyclic();
+
+ return ret;
} else {
initialize_2nd_bitmap(&bitmap2);
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH V3 5/5] Add support for --splitblock-size
2014-10-30 9:45 [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
` (3 preceding siblings ...)
2014-10-30 9:45 ` [PATCH V3 4/5] Add module of calculating start_pfn and end_pfn in each dumpfile Zhou Wenjian
@ 2014-10-30 9:45 ` Zhou Wenjian
2014-11-04 1:51 ` [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time "Zhou, Wenjian/周文剑"
5 siblings, 0 replies; 11+ messages in thread
From: Zhou Wenjian @ 2014-10-30 9:45 UTC (permalink / raw)
To: kexec
Use --splitblock-size to specify splitblock size (KB)
When --split is specified in cyclic mode,splitblock table will be
generated in create_dump_bitmap().
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
makedumpfile.8 | 10 ++++++++++
makedumpfile.c | 4 ++++
makedumpfile.h | 1 +
print_info.c | 5 +++++
4 files changed, 20 insertions(+), 0 deletions(-)
diff --git a/makedumpfile.8 b/makedumpfile.8
index 9cb12c0..5e121fd 100644
--- a/makedumpfile.8
+++ b/makedumpfile.8
@@ -386,6 +386,16 @@ size, so ordinary users don't need to specify this option.
# makedumpfile \-\-cyclic\-buffer 1024 \-d 31 \-x vmlinux /proc/vmcore dumpfile
.TP
+\fB\-\-splitblock\-size\fR \fIsplitblock_size\fR
+Specify the splitblock size in kilo bytes for analysis in the cyclic mode with --split.
+If --splitblock N is specified, difference of each splitted dumpfile size is at most N
+kilo bytes.
+.br
+.B Example:
+.br
+# makedumpfile \-\-splitblock\-size 1024 \-d 31 \-x vmlinux \-\-split /proc/vmcore dumpfile1 dumpfile2
+
+.TP
\fB\-\-non\-cyclic\fR
Running in the non-cyclic mode, this mode uses the old filtering logic same as v1.4.4 or before.
If you feel the cyclic mode is too slow, please try this mode.
diff --git a/makedumpfile.c b/makedumpfile.c
index 1468566..ec712d2 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -9568,6 +9568,7 @@ static struct option longopts[] = {
{"eppic", required_argument, NULL, OPT_EPPIC},
{"non-mmap", no_argument, NULL, OPT_NON_MMAP},
{"mem-usage", no_argument, NULL, OPT_MEM_USAGE},
+ {"splitblock-size", required_argument, NULL, OPT_SPLITBLOCK_SIZE},
{0, 0, 0, 0}
};
@@ -9708,6 +9709,9 @@ main(int argc, char *argv[])
case OPT_CYCLIC_BUFFER:
info->bufsize_cyclic = atoi(optarg);
break;
+ case OPT_SPLITBLOCK_SIZE:
+ info->splitblock_size = atoi(optarg);
+ break;
case '?':
MSG("Commandline parameter is invalid.\n");
MSG("Try `makedumpfile --help' for more information.\n");
diff --git a/makedumpfile.h b/makedumpfile.h
index 39c49b8..d195069 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1885,6 +1885,7 @@ struct elf_prstatus {
#define OPT_EPPIC OPT_START+12
#define OPT_NON_MMAP OPT_START+13
#define OPT_MEM_USAGE OPT_START+14
+#define OPT_SPLITBLOCK_SIZE OPT_START+15
/*
* Function Prototype.
diff --git a/print_info.c b/print_info.c
index f6342d3..16830b2 100644
--- a/print_info.c
+++ b/print_info.c
@@ -203,6 +203,11 @@ print_usage(void)
MSG(" By default, BUFFER_SIZE will be calculated automatically depending on\n");
MSG(" system memory size, so ordinary users don't need to specify this option.\n");
MSG("\n");
+ MSG(" [--splitblock-size SPLITBLOCK_SIZE]:\n");
+ MSG(" Specify the splitblock size in kilo bytes for analysis in the cyclic mode\n");
+ MSG(" with --split.If --splitblock N is specified, difference of each splitted\n");
+ MSG(" dumpfile size is at most N kilo bytes.\n");
+ MSG("\n");
MSG(" [--non-cyclic]:\n");
MSG(" Running in the non-cyclic mode, this mode uses the old filtering logic\n");
MSG(" same as v1.4.4 or before.\n");
--
1.7.1
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time
2014-10-30 9:45 [PATCH V3 0/5] makedumpfile: --split: assign fair I/O workloads in appropriate time Zhou Wenjian
` (4 preceding siblings ...)
2014-10-30 9:45 ` [PATCH V3 5/5] Add support for --splitblock-size Zhou Wenjian
@ 2014-11-04 1:51 ` "Zhou, Wenjian/周文剑"
5 siblings, 0 replies; 11+ messages in thread
From: "Zhou, Wenjian/周文剑" @ 2014-11-04 1:51 UTC (permalink / raw)
To: kexec
Ping...
On 10/30/2014 05:45 PM, Zhou Wenjian wrote:
> v2->v3:
> 1.remove the filtering for incomplete block (previous second pass) and adjust
> relevant codes
> 2.address HATAYAMA's comments about coding style
>
> v1->v2:
> 1.use splitblock instead of block
> 2.add restriction (align to the page size) to splitblock size
> 3.adjust the position of prepare_splitblock_table and check the return code
> 4.use --splitblock-size to specify splitblock size and modify the print_info.c
>
> the v2 test result is:
>
> cyc-buf 128 256 512 1024 2048 4096
> splblk-size
> 2M 24.05(0.65) 24.04(0.65) 24.58(0.65) 24.15(0.64) 24.14(0.64) 49.05(25.46)
> 4M 23.98(0.65) 24.01(0.65) 24.29(0.65) 24.20(0.64) 24.18(0.66) 49.04(25.46)
> 8M 24.02(0.65) 24.03(0.65) 24.25(0.65) 24.26(0.70) 24.15(0.64) 48.98(25.44)
> 16M 24.01(0.65) 24.01(0.65) 24.30(0.65) 24.19(0.64) 24.12(0.65) 48.99(25.45)
> 32M 23.97(0.65) 24.06(0.73) 24.23(0.65) 24.17(0.64) 24.19(0.64) 48.97(25.50)
> 64M 24.06(0.66) 24.07(0.66) 24.27(0.66) 24.16(0.65) 24.17(0.65) 48.98(25.49)
> 128M 24.03(0.67) 24.00(0.67) 24.27(0.66) 24.22(0.66) 24.19(0.66) 48.98(25.48)
> 256M 24.12(0.67) 23.99(0.67) 24.27(0.67) 24.17(0.66) 24.12(0.66) 49.04(25.49)
> 512M 24.06(0.70) 24.08(0.70) 24.26(0.70) 24.14(0.71) 24.19(0.70) 49.13(25.64)
> 1G 24.20(0.82) 24.13(0.81) 24.36(0.81) 24.31(0.80) 24.33(0.81) 49.28(25.75)
> 2G 24.19(0.81) 24.22(0.81) 24.37(0.81) 24.29(0.80) 24.28(0.82) 49.30(25.78)
> 4G 25.29(1.90) 25.26(1.91) 25.49(1.91) 25.41(1.89) 25.50(1.90) 49.99(26.45)
> 8G 25.33(1.90) 26.60(3.23) 26.87(3.21) 26.71(3.23) 26.64(3.22) 51.27(27.73)
> 16G 25.28(1.90) 26.52(3.21) 29.47(5.86) 29.34(5.84) 29.38(5.86) 53.99(30.40)
>
>
> the latest test result turns to:
>
> cyc-buf 128 256 512 1024 2048 4096
> splblk-size
> 2M 23.34(0.00) 23.36(0.00) 23.51(0.00) 23.86(0.00) 23.48(0.00) 23.45(0.00)
> 4M 23.36(0.00) 23.32(0.00) 23.62(0.00) 23.48(0.00) 23.53(0.00) 23.45(0.00)
> 8M 23.33(0.00) 23.46(0.00) 23.61(0.00) 23.77(0.00) 23.52(0.00) 23.53(0.00)
> 16M 23.33(0.00) 23.38(0.00) 23.57(0.00) 23.47(0.00) 23.55(0.00) 23.49(0.00)
> 32M 23.40(0.00) 23.31(0.00) 23.64(0.00) 23.51(0.00) 23.50(0.00) 23.55(0.00)
> 64M 23.39(0.00) 23.41(0.00) 23.62(0.00) 23.47(0.00) 23.46(0.00) 23.50(0.00)
> 128M 23.36(0.00) 23.34(0.00) 23.59(0.00) 23.46(0.00) 23.46(0.00) 23.50(0.00)
> 256M 23.32(0.00) 23.36(0.00) 23.57(0.00) 23.49(0.00) 23.45(0.00) 23.52(0.00)
> 512M 23.32(0.00) 23.32(0.00) 23.63(0.00) 23.49(0.00) 23.52(0.00) 23.51(0.00)
> 1G 23.38(0.00) 23.32(0.00) 23.60(0.00) 23.54(0.00) 23.49(0.00) 23.54(0.00)
> 2G 23.34(0.00) 23.36(0.00) 23.62(0.00) 23.52(0.00) 23.52(0.00) 23.50(0.00)
> 4G 23.36(0.00) 23.37(0.00) 23.60(0.00) 23.48(0.00) 23.56(0.00) 23.55(0.00)
> 8G 23.37(0.00) 23.39(0.00) 23.52(0.00) 23.53(0.00) 23.46(0.00) 23.48(0.00)
> 16G 23.39(0.00) 23.41(0.00) 23.85(0.00) 23.48(0.00) 23.50(0.00) 23.45(0.00)
>
> Zhou Wenjian (5):
> Add support for splitblock
> Add tools for reading and writing from splitblock table
> Add module of generating table
> Add module of calculating start_pfn and end_pfn in each dumpfile
> Add support for --splitblock-size
>
> makedumpfile.8 | 10 +++
> makedumpfile.c | 244 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
> makedumpfile.h | 19 +++++
> print_info.c | 5 +
> 4 files changed, 272 insertions(+), 6 deletions(-)
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 11+ messages in thread