[PATCH v3 0/3] make incomplete dumpfile analyzable

[PATCH v3 0/3] make incomplete dumpfile analyzable

[Zhou Wenjian]

v2->v3:
	remove the part of changing PT_LOAD header
	add check_and_modify_headers()

Wang Xiao (3):
  makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public
  make the incomplete dumpfile generated by ENOSPC error analyzable
  implementation of dealing with kdump-compressed dumpfile with ENOSPC
    error

 diskdump_mod.h |    2 +
 elf_info.c     |   71 ++++++++++----------
 elf_info.h     |    2 +
 makedumpfile.c |  195 +++++++++++++++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 225 insertions(+), 45 deletions(-)


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v3 1/3] makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public

[Zhou Wenjian]

From: Wang Xiao <wangx.fnst@cn.fujitsu.com>

Move the following two functions from internal function to external
function.

get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr)
get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr)

Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
---
 elf_info.c |   71 +++++++++++++++++++++++++++++------------------------------
 elf_info.h |    2 +
 2 files changed, 37 insertions(+), 36 deletions(-)

diff --git a/elf_info.c b/elf_info.c
index 5be1990..24936d4 100644
--- a/elf_info.c
+++ b/elf_info.c
@@ -95,42 +95,6 @@ static unsigned long		size_xen_crash_info;
  * Internal functions.
  */
 static int
-get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr)
-{
-	off_t offset;
-
-	offset = sizeof(Elf64_Ehdr) + sizeof(Elf64_Phdr) * index;
-
-	if (lseek(fd, offset, SEEK_SET) < 0) {
-		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	if (read(fd, phdr, sizeof(Elf64_Phdr)) != sizeof(Elf64_Phdr)) {
-		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	return TRUE;
-}
-
-static int
-get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr)
-{
-	off_t offset;
-
-	offset = sizeof(Elf32_Ehdr) + sizeof(Elf32_Phdr) * index;
-
-	if (lseek(fd, offset, SEEK_SET) < 0) {
-		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	if (read(fd, phdr, sizeof(Elf32_Phdr)) != sizeof(Elf32_Phdr)) {
-		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	return TRUE;
-}
-
-static int
 check_elf_format(int fd, char *filename, int *phnum, unsigned int *num_load)
 {
 	int i;
@@ -446,6 +410,41 @@ int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len)
 /*
  * External functions.
  */
+int
+get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr)
+{
+	off_t offset;
+
+	offset = sizeof(Elf64_Ehdr) + sizeof(Elf64_Phdr) * index;
+
+	if (lseek(fd, offset, SEEK_SET) < 0) {
+		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	if (read(fd, phdr, sizeof(Elf64_Phdr)) != sizeof(Elf64_Phdr)) {
+		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	return TRUE;
+}
+
+int
+get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr)
+{
+	off_t offset;
+
+	offset = sizeof(Elf32_Ehdr) + sizeof(Elf32_Phdr) * index;
+
+	if (lseek(fd, offset, SEEK_SET) < 0) {
+		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	if (read(fd, phdr, sizeof(Elf32_Phdr)) != sizeof(Elf32_Phdr)) {
+		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	return TRUE;
+}
 
 /*
  * Convert Physical Address to File Offset.
diff --git a/elf_info.h b/elf_info.h
index cbabf8a..e712253 100644
--- a/elf_info.h
+++ b/elf_info.h
@@ -27,6 +27,8 @@
 
 #define MAX_SIZE_NHDR	MAX(sizeof(Elf64_Nhdr), sizeof(Elf32_Nhdr))
 
+int get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr);
+int get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr);
 
 off_t paddr_to_offset(unsigned long long paddr);
 off_t paddr_to_offset2(unsigned long long paddr, off_t hint);
-- 
1.7.1


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v3 2/3] make the incomplete dumpfile generated by ENOSPC error analyzable

[Zhou Wenjian]

From: Wang Xiao <wangx.fnst@cn.fujitsu.com>

Since the incomplete dumpfile generated by ENOSPC error can't be anylyzed
by crash utility, but sometimes this file may contain important information
and the panic problem won't be reproduced, then we came up with an idea to
modify the exist data of the incomplete dumpfile to make it analyzable by
crash utility. And each of those dumpfiles has a flag to indicate that it
has been modified. As there are two formats of these dumpfiles, different
methods and flags are needed to deal with each of them,

elf:
This format use the "e_flags" of "elf header" to indicate that it has been
modified.

The method of dealing with "kdump-compressed" format dumpfiles is
implemented
in the next patch.

Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
Singed-of-by: Zhou Wenjian <zhouwj-fnst@cn.fujitsu.com>
---
 makedumpfile.c |  115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 111 insertions(+), 4 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index b4d43d8..141d290 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -24,6 +24,7 @@
 #include <ctype.h>
 #include <sys/time.h>
 #include <limits.h>
+#include <assert.h>
 
 struct symbol_table	symbol_table;
 struct size_table	size_table;
@@ -3768,6 +3769,93 @@ read_flat_data_header(struct makedumpfile_data_header *fdh)
 }
 
 int
+reserve_diskspace(int fd, off_t start_offset, off_t end_offset, char *file_name)
+{
+	size_t buf_size;
+	char *buf = NULL;
+
+	int ret = FALSE;
+
+	assert(start_offset < end_offset);
+	buf_size = end_offset - start_offset;
+
+	if ((buf = malloc(buf_size)) == NULL) {
+		ERRMSG("Can't allocate memory for the size of reserved diskspace. %s\n",
+		       strerror(errno));
+		return FALSE;
+	}
+
+	memset(buf, 0, buf_size);
+	if (!write_buffer(fd, start_offset, buf, buf_size, file_name))
+		goto out;
+
+	ret = TRUE;
+out:
+	if (buf != NULL) {
+		free(buf);
+	}
+
+	return ret;
+}
+
+#define DUMP_ELF_INCOMPLETE 0x1
+int
+check_and_modify_elf_headers(char *filename)
+{
+	int fd, ret = FALSE;
+	Elf64_Ehdr ehdr64;
+	Elf32_Ehdr ehdr32;
+
+	if ((fd = open(filename, O_RDWR)) < 0) {
+		ERRMSG("Can't open the dump file(%s). %s\n",
+		       filename, strerror(errno));
+		return FALSE;
+	}
+
+	/*
+	 * the is_elf64_memory() function still can be used.
+	 */
+	/*
+	 * Set the incomplete flag to the e_flags of elf header.
+	 */
+	if (is_elf64_memory()) { /* ELF64 */
+		if (!get_elf64_ehdr(fd, filename, &ehdr64)) {
+			ERRMSG("Can't get ehdr64.\n");
+			goto out_close_file;
+		}
+		ehdr64.e_flags |= DUMP_ELF_INCOMPLETE;
+		if (!write_buffer(fd, 0, &ehdr64, sizeof(Elf64_Ehdr), filename))
+			goto out_close_file;
+
+	} else { /* ELF32 */
+		if (!get_elf32_ehdr(fd, filename, &ehdr32)) {
+			ERRMSG("Can't get ehdr32.\n");
+			goto out_close_file;
+		}
+		ehdr32.e_flags |= DUMP_ELF_INCOMPLETE;
+		if (!write_buffer(fd, 0, &ehdr32, sizeof(Elf32_Ehdr), filename))
+			goto out_close_file;
+
+	}
+	ret = TRUE;
+out_close_file:
+	if (close(fd) < 0) {
+		ERRMSG("Can't close the dump file(%s). %s\n",
+		       filename, strerror(errno));
+	}
+	return ret;
+}
+
+int
+check_and_modify_headers()
+{
+	if (info->flag_elf_dumpfile)
+		return check_and_modify_elf_headers(info->name_dumpfile);
+	return FALSE;
+}
+
+
+int
 rearrange_dumpdata(void)
 {
 	int read_size, tmp_read_size;
@@ -5498,6 +5586,13 @@ write_elf_header(struct cache_data *cd_header)
 	size_note          = note.p_filesz;
 
 	/*
+	 * Reserve a space to store the whole program headers.
+	 */
+	if (!reserve_diskspace(cd_header->fd, cd_header->offset,
+				offset_note_dumpfile, cd_header->file_name))
+		goto out;
+
+	/*
 	 * Modify the note size in PT_NOTE header to accomodate eraseinfo data.
 	 * Eraseinfo will be written later.
 	 */
@@ -8015,10 +8110,10 @@ writeout_dumpfile(void)
 			goto out;
 		if (info->flag_cyclic) {
 			if (!write_elf_pages_cyclic(&cd_header, &cd_page))
-				goto out;
+				goto write_cache_enospc;
 		} else {
 			if (!write_elf_pages(&cd_header, &cd_page))
-				goto out;
+				goto write_cache_enospc;
 		}
 		if (!write_elf_eraseinfo(&cd_header))
 			goto out;
@@ -8045,6 +8140,11 @@ writeout_dumpfile(void)
 	}
 
 	ret = TRUE;
+write_cache_enospc:
+	if ((ret == FALSE) && info->flag_nospace && !info->flag_flatten) {
+		if (!write_cache_bufsz(&cd_header))
+			ERRMSG("This dumpfile may lost some important data.\n");
+	}
 out:
 	free_cache_data(&cd_header);
 	free_cache_data(&cd_page);
@@ -8237,8 +8337,15 @@ retry:
 		 * to create a dumpfile with it again.
 		 */
 		num_retry++;
-		if ((info->dump_level = get_next_dump_level(num_retry)) < 0)
- 			return FALSE;
+		if ((info->dump_level = get_next_dump_level(num_retry)) < 0) {
+			if (!info->flag_flatten) {
+				if (check_and_modify_headers())
+					MSG("This is an incomplete dumpfile,"
+						" but might analyzable.\n");
+			}
+
+			return FALSE;
+		}
 		MSG("Retry to create a dumpfile by dump_level(%d).\n",
 		    info->dump_level);
 		if (!delete_dumpfile())
-- 
1.7.1


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH v3 3/3] implementation of dealing with kdump-compressed dumpfile with ENOSPC error

[Zhou Wenjian]

From: Wang Xiao <wangx.fnst@cn.fujitsu.com>

kdump-compressed:
Dump the bitmap before any page header and page data. This format use
"status" of "disk_dump_header" to indicate that it has been modified.

Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
---
 diskdump_mod.h |    2 +
 makedumpfile.c |   80 ++++++++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 77 insertions(+), 5 deletions(-)

diff --git a/diskdump_mod.h b/diskdump_mod.h
index dd24eb2..275ed83 100644
--- a/diskdump_mod.h
+++ b/diskdump_mod.h
@@ -95,6 +95,8 @@ struct kdump_sub_header {
 #define DUMP_DH_COMPRESSED_LZO	0x2	/* paged is compressed with lzo */
 #define DUMP_DH_COMPRESSED_SNAPPY	0x4
 					/* paged is compressed with snappy */
+#define DUMP_DH_COMPRESSED_INCOMPLETE	0x8
+					/* indicate an incomplete dumpfile */
 
 /* descriptor of each page for vmcore */
 typedef struct page_desc {
diff --git a/makedumpfile.c b/makedumpfile.c
index 141d290..4dba83e 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -3847,10 +3847,80 @@ out_close_file:
 }
 
 int
+check_and_modify_kdump_headers(char *filename) {
+	int fd, ret = FALSE;
+	struct disk_dump_header dh;
+
+	if (!read_disk_dump_header(&dh, filename))
+		return FALSE;
+
+	if ((fd = open(filename, O_RDWR)) < 0) {
+		ERRMSG("Can't open the dump file(%s). %s\n",
+		       filename, strerror(errno));
+		return FALSE;
+	}
+
+	/*
+	 * Set the incomplete flag to the status of disk_dump_header.
+	 */
+	dh.status |= DUMP_DH_COMPRESSED_INCOMPLETE;
+
+	/*
+	 * It's safe to overwrite the disk_dump_header.
+	 */
+	if (!write_buffer(fd, 0, &dh, sizeof(struct disk_dump_header), filename))
+		goto out_close_file;
+
+	ret = TRUE;
+out_close_file:
+	if (close(fd) < 0) {
+		ERRMSG("Can't close the dump file(%s). %s\n",
+		       filename, strerror(errno));
+	}
+
+	return ret;
+}
+
+int
+check_and_modify_multiple_kdump_headers() {
+	int i, status, ret = TRUE;
+	pid_t pid;
+	pid_t array_pid[info->num_dumpfile];
+
+	for (i = 0; i < info->num_dumpfile; i++) {
+		if ((pid = fork()) < 0) {
+			return FALSE;
+
+		} else if (pid == 0) { /* Child */
+			if (!check_and_modify_kdump_headers(SPLITTING_DUMPFILE(i)))
+				exit(1);
+			exit(0);
+		}
+		array_pid[i] = pid;
+	}
+
+	for (i = 0; i < info->num_dumpfile; i++) {
+		waitpid(array_pid[i], &status, WUNTRACED);
+		if (!WIFEXITED(status) || WEXITSTATUS(status) == 1) {
+			ERRMSG("Check and modify the incomplete dumpfile(%s) failed.\n",
+			       SPLITTING_DUMPFILE(i));
+			ret = FALSE;
+		}
+	}
+
+	return ret;
+}
+
+int
 check_and_modify_headers()
 {
 	if (info->flag_elf_dumpfile)
 		return check_and_modify_elf_headers(info->name_dumpfile);
+	else
+		if(info->flag_split)
+			return check_and_modify_multiple_kdump_headers();
+		else
+			return check_and_modify_kdump_headers(info->name_dumpfile);
 	return FALSE;
 }
 
@@ -7123,11 +7193,11 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
 		if (!exclude_unnecessary_pages_cyclic(&cycle))
 			return FALSE;
 
-		if (!write_kdump_pages_cyclic(cd_header, cd_page, &pd_zero,
-					&offset_data, &cycle))
+		if (!write_kdump_bitmap2_cyclic(&cycle))
 			return FALSE;
 
-		if (!write_kdump_bitmap2_cyclic(&cycle))
+		if (!write_kdump_pages_cyclic(cd_header, cd_page, &pd_zero,
+					&offset_data, &cycle))
 			return FALSE;
 	}
 
@@ -8127,12 +8197,12 @@ writeout_dumpfile(void)
 	} else {
 		if (!write_kdump_header())
 			goto out;
+		if (!write_kdump_bitmap())
+			goto out;
 		if (!write_kdump_pages(&cd_header, &cd_page))
 			goto out;
 		if (!write_kdump_eraseinfo(&cd_page))
 			goto out;
-		if (!write_kdump_bitmap())
-			goto out;
 	}
 	if (info->flag_flatten) {
 		if (!write_end_flat_header())
-- 
1.7.1


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v3 0/3] make incomplete dumpfile analyzable

["Zhou, Wenjian/周文剑"]

I will continue these patches.

Zhou Wenjian

On 10/14/2014 09:17 AM, Zhou Wenjian wrote:
> v2->v3:
> 	remove the part of changing PT_LOAD header
> 	add check_and_modify_headers()
>
> Wang Xiao (3):
>    makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public
>    make the incomplete dumpfile generated by ENOSPC error analyzable
>    implementation of dealing with kdump-compressed dumpfile with ENOSPC
>      error
>
>   diskdump_mod.h |    2 +
>   elf_info.c     |   71 ++++++++++----------
>   elf_info.h     |    2 +
>   makedumpfile.c |  195 +++++++++++++++++++++++++++++++++++++++++++++++++++++---
>   4 files changed, 225 insertions(+), 45 deletions(-)
>
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

RE: [PATCH v3 0/3] make incomplete dumpfile analyzable

[Atsushi Kumagai]

Hello Zhou,

>I will continue these patches.

You meant you will fix the reassemble process to consider
the incomplete flags of each split dump ?


Thanks,
Atsushi Kumagai

>
>Zhou Wenjian
>
>On 10/14/2014 09:17 AM, Zhou Wenjian wrote:
>> v2->v3:
>> 	remove the part of changing PT_LOAD header
>> 	add check_and_modify_headers()
>>
>> Wang Xiao (3):
>>    makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public
>>    make the incomplete dumpfile generated by ENOSPC error analyzable
>>    implementation of dealing with kdump-compressed dumpfile with ENOSPC
>>      error
>>
>>   diskdump_mod.h |    2 +
>>   elf_info.c     |   71 ++++++++++----------
>>   elf_info.h     |    2 +
>>   makedumpfile.c |  195 +++++++++++++++++++++++++++++++++++++++++++++++++++++---
>>   4 files changed, 225 insertions(+), 45 deletions(-)
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
>
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH v3 0/3] make incomplete dumpfile analyzable

["Zhou, Wenjian/周文剑"]

On 10/20/2014 09:18 AM, Atsushi Kumagai wrote:
> You meant you will fix the reassemble process to consider
> the incomplete flags of each split dump ?
Sorry, I have fixed it, but forgot to commit. I will resend it.

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec