* [PATCH / RFC] makedumpfile header to show LZO/snappy/zlib [not found] <1632896047.12077061.1378995804236.JavaMail.root@redhat.com> @ 2013-09-12 14:35 ` Dave Anderson 2013-09-20 4:21 ` Atsushi Kumagai 0 siblings, 1 reply; 5+ messages in thread From: Dave Anderson @ 2013-09-12 14:35 UTC (permalink / raw) To: kexec [-- Attachment #1: Type: text/plain, Size: 1758 bytes --] With the advent of LZO and snappy compression support, it would be helpful if it could be readily determined which type of compression was used to create the dumpfile. The zlib/LZO/snappy indicator is only found down in the bowels of each page_desc structure: /* page flags */ #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib */ #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo */ #define DUMP_DH_COMPRESSED_SNAPPY 0x4 /* paged is compressed with snappy */ /* descriptor of each page for vmcore */ typedef struct page_desc { off_t offset; /* the offset of the page data*/ unsigned int size; /* the size of this dump page */ unsigned int flags; /* flags */ unsigned long long page_flags; /* page flags */ } page_desc_t; The compressed kdump's header does contain a "status" flag: struct disk_dump_header { char signature[SIG_LEN]; /* = "KDUMP " */ int header_version; /* Dump header version */ struct new_utsname utsname; /* copy of system_utsname */ struct timeval timestamp; /* Time stamp */ unsigned int status; /* Above flags */ ... which has these 3 flags: #define DUMP_HEADER_COMPLETED 0 #define DUMP_HEADER_INCOMPLETED 1 #define DUMP_HEADER_COMPRESSED 8 But they are all remnants of the old "diskdump" facility, and none of them are ever used by makedumpfile. Any chance that the LZO/snappy/zlib method could also be put up in the disk_dump_header? Dave [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: compression_header.patch --] [-- Type: text/x-patch; name=compression_header.patch, Size: 796 bytes --] --- makedumpfile-1.5.4/makedumpfile.c.orig +++ makedumpfile-1.5.4/makedumpfile.c @@ -5162,6 +5162,16 @@ write_kdump_header(void) dh->bitmap_blocks = divideup(info->len_bitmap, dh->block_size); memcpy(&dh->timestamp, &info->timestamp, sizeof(dh->timestamp)); memcpy(&dh->utsname, &info->system_utsname, sizeof(dh->utsname)); + if (info->flag_compress & DUMP_DH_COMPRESSED_ZLIB) + dh->status |= DUMP_DH_COMPRESSED_ZLIB; +#ifdef USELZO + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) + dh->status |= DUMP_DH_COMPRESSED_LZO) +#endif +#ifdef USESNAPPY + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) +#endif size = sizeof(struct disk_dump_header); if (!write_buffer(info->fd_dumpfile, 0, dh, size, info->name_dumpfile)) [-- Attachment #3: Type: text/plain, Size: 143 bytes --] _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH / RFC] makedumpfile header to show LZO/snappy/zlib 2013-09-12 14:35 ` [PATCH / RFC] makedumpfile header to show LZO/snappy/zlib Dave Anderson @ 2013-09-20 4:21 ` Atsushi Kumagai 2013-09-20 13:04 ` Dave Anderson 0 siblings, 1 reply; 5+ messages in thread From: Atsushi Kumagai @ 2013-09-20 4:21 UTC (permalink / raw) To: anderson; +Cc: kexec Hello Dave, (2013/09/16 0:56), Dave Anderson wrote: > > With the advent of LZO and snappy compression support, it would be > helpful if it could be readily determined which type of compression > was used to create the dumpfile. > > The zlib/LZO/snappy indicator is only found down in the bowels of each > page_desc structure: > > /* page flags */ > #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib */ > #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo */ > #define DUMP_DH_COMPRESSED_SNAPPY 0x4 > /* paged is compressed with snappy */ > > /* descriptor of each page for vmcore */ > typedef struct page_desc { > off_t offset; /* the offset of the page data*/ > unsigned int size; /* the size of this dump page */ > unsigned int flags; /* flags */ > unsigned long long page_flags; /* page flags */ > } page_desc_t; > > The compressed kdump's header does contain a "status" flag: > > struct disk_dump_header { > char signature[SIG_LEN]; /* = "KDUMP " */ > int header_version; /* Dump header version */ > struct new_utsname utsname; /* copy of system_utsname */ > struct timeval timestamp; /* Time stamp */ > unsigned int status; /* Above flags */ > ... > > which has these 3 flags: > > #define DUMP_HEADER_COMPLETED 0 > #define DUMP_HEADER_INCOMPLETED 1 > #define DUMP_HEADER_COMPRESSED 8 > > But they are all remnants of the old "diskdump" facility, and none of them > are ever used by makedumpfile. > > Any chance that the LZO/snappy/zlib method could also be put up in > the disk_dump_header? > > Dave > > --- makedumpfile-1.5.4/makedumpfile.c.orig > +++ makedumpfile-1.5.4/makedumpfile.c > @@ -5162,6 +5162,16 @@ write_kdump_header(void) > dh->bitmap_blocks = divideup(info->len_bitmap, dh->block_size); > memcpy(&dh->timestamp, &info->timestamp, sizeof(dh->timestamp)); > memcpy(&dh->utsname, &info->system_utsname, sizeof(dh->utsname)); > + if (info->flag_compress & DUMP_DH_COMPRESSED_ZLIB) > + dh->status |= DUMP_DH_COMPRESSED_ZLIB; > +#ifdef USELZO > + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) > + dh->status |= DUMP_DH_COMPRESSED_LZO) > +#endif > +#ifdef USESNAPPY > + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) > + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) > +#endif dh->status will not guarantee that *all pages* are compressed by the check below: > #ifdef USELZO > } else if (info->flag_lzo_support > && (info->flag_compress & DUMP_DH_COMPRESSED_LZO) > && ((size_out = info->page_size), > lzo1x_1_compress(buf, info->page_size, buf_out, > &size_out, wrkmem) == LZO_E_OK) > && (size_out < info->page_size)) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^ When there is no effect of compression, the page will not be compressed. > pd.flags = DUMP_DH_COMPRESSED_LZO; > pd.size = size_out; > memcpy(buf, buf_out, pd.size); > #endif If we are on the same page, I'll accept your patch. Thanks Atsushi Kumagai > > size = sizeof(struct disk_dump_header); > if (!write_buffer(info->fd_dumpfile, 0, dh, size, info->name_dumpfile)) _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH / RFC] makedumpfile header to show LZO/snappy/zlib 2013-09-20 4:21 ` Atsushi Kumagai @ 2013-09-20 13:04 ` Dave Anderson 2013-09-26 13:31 ` Petr Tesarik 0 siblings, 1 reply; 5+ messages in thread From: Dave Anderson @ 2013-09-20 13:04 UTC (permalink / raw) To: Atsushi Kumagai; +Cc: kexec ----- Original Message ----- > Hello Dave, > > (2013/09/16 0:56), Dave Anderson wrote: > > > > With the advent of LZO and snappy compression support, it would be > > helpful if it could be readily determined which type of compression > > was used to create the dumpfile. > > > > The zlib/LZO/snappy indicator is only found down in the bowels of each > > page_desc structure: > > > > /* page flags */ > > #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib > > */ > > #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo > > */ > > #define DUMP_DH_COMPRESSED_SNAPPY 0x4 > > /* paged is compressed with > > snappy */ > > > > /* descriptor of each page for vmcore */ > > typedef struct page_desc { > > off_t offset; /* the offset of the > > page data*/ > > unsigned int size; /* the size of this dump > > page */ > > unsigned int flags; /* flags */ > > unsigned long long page_flags; /* page flags */ > > } page_desc_t; > > > > The compressed kdump's header does contain a "status" flag: > > > > struct disk_dump_header { > > char signature[SIG_LEN]; /* = "KDUMP " > > */ > > int header_version; /* Dump header version > > */ > > struct new_utsname utsname; /* copy of > > system_utsname */ > > struct timeval timestamp; /* Time stamp */ > > unsigned int status; /* Above flags */ > > ... > > > > which has these 3 flags: > > > > #define DUMP_HEADER_COMPLETED 0 > > #define DUMP_HEADER_INCOMPLETED 1 > > #define DUMP_HEADER_COMPRESSED 8 > > > > But they are all remnants of the old "diskdump" facility, and none of them > > are ever used by makedumpfile. > > > > Any chance that the LZO/snappy/zlib method could also be put up in > > the disk_dump_header? > > > > Dave > > > > --- makedumpfile-1.5.4/makedumpfile.c.orig > > +++ makedumpfile-1.5.4/makedumpfile.c > > @@ -5162,6 +5162,16 @@ write_kdump_header(void) > > dh->bitmap_blocks = divideup(info->len_bitmap, dh->block_size); > > memcpy(&dh->timestamp, &info->timestamp, sizeof(dh->timestamp)); > > memcpy(&dh->utsname, &info->system_utsname, sizeof(dh->utsname)); > > + if (info->flag_compress & DUMP_DH_COMPRESSED_ZLIB) > > + dh->status |= DUMP_DH_COMPRESSED_ZLIB; > > +#ifdef USELZO > > + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) > > + dh->status |= DUMP_DH_COMPRESSED_LZO) > > +#endif > > +#ifdef USESNAPPY > > + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) > > + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) > > +#endif > > dh->status will not guarantee that *all pages* are compressed > by the check below: > > > #ifdef USELZO > > } else if (info->flag_lzo_support > > && (info->flag_compress & > > DUMP_DH_COMPRESSED_LZO) > > && ((size_out = info->page_size), > > lzo1x_1_compress(buf, info->page_size, > > buf_out, > > &size_out, wrkmem) == > > LZO_E_OK) > > && (size_out < info->page_size)) { > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > When there is no effect of compression, the page will not be compressed. > > > pd.flags = DUMP_DH_COMPRESSED_LZO; > > pd.size = size_out; > > memcpy(buf, buf_out, pd.size); > > #endif > > If we are on the same page, I'll accept your patch. Right, I understand. Thanks, Dave > > > Thanks > Atsushi Kumagai > > > > > size = sizeof(struct disk_dump_header); > > if (!write_buffer(info->fd_dumpfile, 0, dh, size, > > info->name_dumpfile)) > _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH / RFC] makedumpfile header to show LZO/snappy/zlib 2013-09-20 13:04 ` Dave Anderson @ 2013-09-26 13:31 ` Petr Tesarik 2013-09-26 17:05 ` Dave Anderson 0 siblings, 1 reply; 5+ messages in thread From: Petr Tesarik @ 2013-09-26 13:31 UTC (permalink / raw) To: Dave Anderson; +Cc: kexec, Atsushi Kumagai On Fri, 20 Sep 2013 09:04:40 -0400 (EDT) Dave Anderson <anderson@redhat.com> wrote: > > > ----- Original Message ----- > > Hello Dave, > > > > (2013/09/16 0:56), Dave Anderson wrote: > > > > > > With the advent of LZO and snappy compression support, it would be > > > helpful if it could be readily determined which type of compression > > > was used to create the dumpfile. > > > > > > The zlib/LZO/snappy indicator is only found down in the bowels of each > > > page_desc structure: > > > > > > /* page flags */ > > > #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib > > > */ > > > #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo > > > */ > > > #define DUMP_DH_COMPRESSED_SNAPPY 0x4 > > > /* paged is compressed with > > > snappy */ > > > > > > /* descriptor of each page for vmcore */ > > > typedef struct page_desc { > > > off_t offset; /* the offset of the > > > page data*/ > > > unsigned int size; /* the size of this dump > > > page */ > > > unsigned int flags; /* flags */ > > > unsigned long long page_flags; /* page flags */ > > > } page_desc_t; > > > > > > The compressed kdump's header does contain a "status" flag: > > > > > > struct disk_dump_header { > > > char signature[SIG_LEN]; /* = "KDUMP " > > > */ > > > int header_version; /* Dump header version > > > */ > > > struct new_utsname utsname; /* copy of > > > system_utsname */ > > > struct timeval timestamp; /* Time stamp */ > > > unsigned int status; /* Above flags */ > > > ... > > > > > > which has these 3 flags: > > > > > > #define DUMP_HEADER_COMPLETED 0 > > > #define DUMP_HEADER_INCOMPLETED 1 > > > #define DUMP_HEADER_COMPRESSED 8 > > > > > > But they are all remnants of the old "diskdump" facility, and none of them > > > are ever used by makedumpfile. > > > > > > Any chance that the LZO/snappy/zlib method could also be put up in > > > the disk_dump_header? > > > > > > Dave > > > > > > --- makedumpfile-1.5.4/makedumpfile.c.orig > > > +++ makedumpfile-1.5.4/makedumpfile.c > > > @@ -5162,6 +5162,16 @@ write_kdump_header(void) > > > dh->bitmap_blocks = divideup(info->len_bitmap, dh->block_size); > > > memcpy(&dh->timestamp, &info->timestamp, sizeof(dh->timestamp)); > > > memcpy(&dh->utsname, &info->system_utsname, sizeof(dh->utsname)); > > > + if (info->flag_compress & DUMP_DH_COMPRESSED_ZLIB) > > > + dh->status |= DUMP_DH_COMPRESSED_ZLIB; > > > +#ifdef USELZO > > > + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) > > > + dh->status |= DUMP_DH_COMPRESSED_LZO) > > > +#endif > > > +#ifdef USESNAPPY > > > + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) > > > + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) > > > +#endif > > > > dh->status will not guarantee that *all pages* are compressed > > by the check below: > > > > > #ifdef USELZO > > > } else if (info->flag_lzo_support > > > && (info->flag_compress & > > > DUMP_DH_COMPRESSED_LZO) > > > && ((size_out = info->page_size), > > > lzo1x_1_compress(buf, info->page_size, > > > buf_out, > > > &size_out, wrkmem) == > > > LZO_E_OK) > > > && (size_out < info->page_size)) { > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > When there is no effect of compression, the page will not be compressed. > > > > > pd.flags = DUMP_DH_COMPRESSED_LZO; > > > pd.size = size_out; > > > memcpy(buf, buf_out, pd.size); > > > #endif > > > > If we are on the same page, I'll accept your patch. > > Right, I understand. I think the primary intention is to let crash determine early if it is able to open a given dump or not. And that's a good idea! From a semantic point of view, the status field in the header should be a logical OR of all compression methods used in the coredump. AFAICS, Dave's patch would actually work for the current implementation. Did I miss something? Petr T _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH / RFC] makedumpfile header to show LZO/snappy/zlib 2013-09-26 13:31 ` Petr Tesarik @ 2013-09-26 17:05 ` Dave Anderson 0 siblings, 0 replies; 5+ messages in thread From: Dave Anderson @ 2013-09-26 17:05 UTC (permalink / raw) To: Petr Tesarik; +Cc: kexec, Atsushi Kumagai ----- Original Message ----- > On Fri, 20 Sep 2013 09:04:40 -0400 (EDT) > Dave Anderson <anderson@redhat.com> wrote: > > > > > > > ----- Original Message ----- > > > Hello Dave, > > > > > > (2013/09/16 0:56), Dave Anderson wrote: > > > > > > > > With the advent of LZO and snappy compression support, it would be > > > > helpful if it could be readily determined which type of compression > > > > was used to create the dumpfile. > > > > > > > > The zlib/LZO/snappy indicator is only found down in the bowels of each > > > > page_desc structure: > > > > > > > > /* page flags */ > > > > #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib > > > > */ > > > > #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo > > > > */ > > > > #define DUMP_DH_COMPRESSED_SNAPPY 0x4 > > > > /* paged is compressed with snappy */ > > > > > > > > /* descriptor of each page for vmcore */ > > > > typedef struct page_desc { > > > > off_t offset; /* the offset of the page data */ > > > > unsigned int size; /* the size of this dump page */ > > > > unsigned int flags; /* flags */ > > > > unsigned long long page_flags; /* page flags */ > > > > } page_desc_t; > > > > > > > > The compressed kdump's header does contain a "status" flag: > > > > > > > > struct disk_dump_header { > > > > char signature[SIG_LEN]; /* = "KDUMP " */ > > > > int header_version; /* Dump header version */ > > > > struct new_utsname utsname; /* copy of system_utsname */ > > > > struct timeval timestamp; /* Time stamp */ > > > > unsigned int status; /* Above flags */ > > > > ... > > > > > > > > which has these 3 flags: > > > > > > > > #define DUMP_HEADER_COMPLETED 0 > > > > #define DUMP_HEADER_INCOMPLETED 1 > > > > #define DUMP_HEADER_COMPRESSED 8 > > > > > > > > But they are all remnants of the old "diskdump" facility, and none of them > > > > are ever used by makedumpfile. > > > > > > > > Any chance that the LZO/snappy/zlib method could also be put up in > > > > the disk_dump_header? > > > > > > > > Dave > > > > > > > > --- makedumpfile-1.5.4/makedumpfile.c.orig > > > > +++ makedumpfile-1.5.4/makedumpfile.c > > > > @@ -5162,6 +5162,16 @@ write_kdump_header(void) > > > > dh->bitmap_blocks = divideup(info->len_bitmap, dh->block_size); > > > > memcpy(&dh->timestamp, &info->timestamp, sizeof(dh->timestamp)); > > > > memcpy(&dh->utsname, &info->system_utsname, sizeof(dh->utsname)); > > > > + if (info->flag_compress & DUMP_DH_COMPRESSED_ZLIB) > > > > + dh->status |= DUMP_DH_COMPRESSED_ZLIB; > > > > +#ifdef USELZO > > > > + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) > > > > + dh->status |= DUMP_DH_COMPRESSED_LZO) > > > > +#endif > > > > +#ifdef USESNAPPY > > > > + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) > > > > + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) > > > > +#endif > > > > > > dh->status will not guarantee that *all pages* are compressed > > > by the check below: > > > > > > > #ifdef USELZO > > > > } else if (info->flag_lzo_support > > > > && (info->flag_compress & > > > > DUMP_DH_COMPRESSED_LZO) > > > > && ((size_out = info->page_size), > > > > lzo1x_1_compress(buf, info->page_size, > > > > buf_out, > > > > &size_out, wrkmem) == > > > > LZO_E_OK) > > > > && (size_out < info->page_size)) { > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > When there is no effect of compression, the page will not be compressed. > > > > > > > pd.flags = DUMP_DH_COMPRESSED_LZO; > > > > pd.size = size_out; > > > > memcpy(buf, buf_out, pd.size); > > > > #endif > > > > > > If we are on the same page, I'll accept your patch. > > > > Right, I understand. > > I think the primary intention is to let crash determine early if it is > able to open a given dump or not. And that's a good idea! The primary intention is informational only, either for "help -[nD]" or for the "crash -d# ..." display which dumps the header contents immediately after reading it. As it is now, if crash is unable to support the compression type because the relevant library was not built in, it will accept the vmcore, but will fail upon the first read attempt with a message "uncompress failed: no lzo compression support" (or snappy). > From a semantic point of view, the status field in the header should be > a logical OR of all compression methods used in the coredump. AFAICS, > Dave's patch would actually work for the current implementation. > > Did I miss something? > > Petr T I don't think so -- Atsushi's point was simply that there is a good probability that there will be a mixture of compressed and uncompressed pages. Dave _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2013-09-26 17:05 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1632896047.12077061.1378995804236.JavaMail.root@redhat.com>
2013-09-12 14:35 ` [PATCH / RFC] makedumpfile header to show LZO/snappy/zlib Dave Anderson
2013-09-20 4:21 ` Atsushi Kumagai
2013-09-20 13:04 ` Dave Anderson
2013-09-26 13:31 ` Petr Tesarik
2013-09-26 17:05 ` Dave Anderson
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox