r: (1, 'Internal error', 'panic: xc_dom_core.c:273: xc_dom_do_gunzip: inflate failed (rc=-3)').. only on Intel hardware and only under 32-bit dom0

r: (1, 'Internal error', 'panic: xc_dom_core.c:273: xc_dom_do_gunzip: inflate failed (rc=-3)').. only on Intel hardware and only under 32-bit dom0

[Konrad Rzeszutek Wilk]

The combination is bit awkward so not sure why this is happening, but with
Xen 4.1.x hypervisor (64-bit), with a 32-bit dom0 (3.3-rcX or 3.2) and with a 3.3-rcX or 3.2 domU
I get this:

sh-4.1# Feb 20 17:47:41 tst006 init: reloading /etc/inittab
xl info
host                   : tst006.dumpdata.com
release                : 3.3.0-rc4
version                : #1 SMP PREEMPT Mi686
nr_cpus                : 2
nr_nodes               : 1
cores_per_socket       : 2
threads_per_core       : 1
cpu_mhz                : 2333
hw_caps                : bfebfbff:20100800:00000000:00000940:0000e3fd:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 4085
free_memory            : 717
free_cpus              : 0
xen_major              : 4
xen_minor              : 1
xen_extra              : -120220
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xff800000
xen_changeset          : Mon Feb 13 17:57:47 2012 +0000 23225:f2543f449a49
xen_commandline        : com1=115200,8n1 cpufreq=verbose apic=debug apic_verbosity=debug console=com1,vga loglvl=all guest_loglvl=all apic=debug
cc_compiler            : gcc version 4.4.4 20100503 (Red Hat 4.4.4-2) (GCC) 
cc_compile_by          : konrad
cc_compile_domain      : dumpdata.com
cc_compile_date        : Mon Feb 20 12:25:42 EST 2012
xend_config_format     : 4
sh-4.1# xm create /mnt/lab/latest/test.xm 
Using config file "/mnt/lab/latest/test.xm".
Error: (1, 'Internal error', 'panic: xc_dom_core.c:273: xc_dom_do_gunzip: inflate failed (rc=-3)')

And only on Intel.. and only if it is a 32-bit dom0. If I do the same test with a 64-bit dom0 I do not see this problem.

Any thoughts of what it might be or what I should try out? My thought
was to swap out the hypervisor (use a Xen 4.0) or unstable and see if I get the same result.
But maybe there is something obvious out there?

Re: r: (1, 'Internal error', 'panic: xc_dom_core.c:273: xc_dom_do_gunzip: inflate failed (rc=-3)').. only on Intel hardware and only under 32-bit dom0

[Ian Campbell]

[-- Attachment #1: Type: text/plain, Size: 1055 bytes --]

On Mon, 2012-02-20 at 18:32 +0000, Konrad Rzeszutek Wilk wrote:
[...]
> sh-4.1# xm create /mnt/lab/latest/test.xm 
> Using config file "/mnt/lab/latest/test.xm".
> Error: (1, 'Internal error', 'panic: xc_dom_core.c:273:
> xc_dom_do_gunzip: inflate failed (rc=-3)')

-3 == ESRCH? Seems unlikely...

Aha, in this context it is the return value of inflate() and therefore
it is Z_DATA_ERROR.

> And only on Intel.. and only if it is a 32-bit dom0. If I do the same
> test with a 64-bit dom0 I do not see this problem.

Different decompression libraries in your root filesystems perhaps? Can
you decompress the bzImage by hand? (perhaps using the attached bzeplode
to extract the compressed data payload)

Does it work with xl?

> Any thoughts of what it might be or what I should try out? My thought
> was to swap out the hypervisor (use a Xen 4.0) or unstable and see if
> I get the same result.

It is unlikely to be the hypervisor, more likely to be one of the
userspace components of the toolstack.

> But maybe there is something obvious out there?
> 


[-- Attachment #2: bzexplode.c --]
[-- Type: text/x-csrc, Size: 1251 bytes --]

#include <stdio.h>
#include <fcntl.h>
#include <stdint.h>
#include <unistd.h>

#include <inttypes.h>

#include <err.h>

#include <sys/types.h>
#include <sys/mman.h>
#include <sys/stat.h>

int main(int argc, char **argv)
{
	int fd;
	struct stat sb;
	void *p;
	uint8_t *hdr;
	int setup_sectors;
	uint32_t compressed_payload_offset;
	uint32_t compressed_payload_length;

	if (argc != 2)
		errx(1, "usage: bzexplode <bzImage>");

	fd = open(argv[1], O_RDONLY);
	if (fd < 0)
		err(1, "open");

	if (fstat(fd, &sb) < 0)
		err(1, "fstat");

	p = mmap(0, sb.st_size, PROT_READ, MAP_SHARED, fd, 0);
	if (p == MAP_FAILED)
		err(1, "mmap");

	hdr = p;
	setup_sectors = hdr[0x1f1];
	compressed_payload_offset = *(uint32_t*)&hdr[0x248];

	fprintf(stderr, "setup sectors %d\n", setup_sectors);

	compressed_payload_offset += (setup_sectors+1) * 512;

	//compressed_payload_length = *(uint32_t*)(p + compressed_payload_offset - 4);
	compressed_payload_length = *(uint32_t*)&hdr[0x24c];

	fprintf(stderr, "compressed_payload_offset %"PRIx32" (abs)\n",
		compressed_payload_offset);
	fprintf(stderr, "compressed_payload_length %"PRIx32"\n",
		compressed_payload_length);

	write(1,
	      p + compressed_payload_offset,
	      compressed_payload_length);
	return 0;
}

[-- Attachment #3: Type: text/plain, Size: 132 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xensource.com/xen-devel

Re: r: (1, 'Internal error', 'panic: xc_dom_core.c:273: xc_dom_do_gunzip: inflate failed (rc=-3)').. only on Intel hardware and only under 32-bit dom0

[Jan Beulich]

>>> On 20.02.12 at 19:32, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> And only on Intel.. and only if it is a 32-bit dom0. If I do the same test 
> with a 64-bit dom0 I do not see this problem.
> 
> Any thoughts of what it might be or what I should try out? My thought
> was to swap out the hypervisor (use a Xen 4.0) or unstable and see if I get 
> the same result.
> But maybe there is something obvious out there?

Did you check that it's not the kernel image itself on that particular box
that's corrupted?

Jan

Re: r: (1, 'Internal error', 'panic: xc_dom_core.c:273: xc_dom_do_gunzip: inflate failed (rc=-3)').. only on Intel hardware and only under 32-bit dom0

[Konrad Rzeszutek Wilk]

On Tue, Feb 21, 2012 at 08:59:35AM +0000, Ian Campbell wrote:
> On Mon, 2012-02-20 at 18:32 +0000, Konrad Rzeszutek Wilk wrote:
> [...]
> > sh-4.1# xm create /mnt/lab/latest/test.xm 
> > Using config file "/mnt/lab/latest/test.xm".
> > Error: (1, 'Internal error', 'panic: xc_dom_core.c:273:
> > xc_dom_do_gunzip: inflate failed (rc=-3)')
> 
> -3 == ESRCH? Seems unlikely...
> 
> Aha, in this context it is the return value of inflate() and therefore
> it is Z_DATA_ERROR.
> 
> > And only on Intel.. and only if it is a 32-bit dom0. If I do the same
> > test with a 64-bit dom0 I do not see this problem.
> 
> Different decompression libraries in your root filesystems perhaps? Can

Hm, not sure. So I tried the code on the root filesystem and it worked
fine and gave an binary image that was the same as on another machine.

But I wonder if the libxl use the shared library or compile it in. Perhaps
there is some weird library version mismatch. Will double-check.

> you decompress the bzImage by hand? (perhaps using the attached bzeplode
> to extract the compressed data payload)
> 
> Does it work with xl?

Nope.
> 
> > Any thoughts of what it might be or what I should try out? My thought
> > was to swap out the hypervisor (use a Xen 4.0) or unstable and see if
> > I get the same result.
> 
> It is unlikely to be the hypervisor, more likely to be one of the
> userspace components of the toolstack.

I think you are right..

Re: r: (1, 'Internal error', 'panic: xc_dom_core.c:273: xc_dom_do_gunzip: inflate failed (rc=-3)').. only on Intel hardware and only under 32-bit dom0

[Konrad Rzeszutek Wilk]

On Tue, Feb 21, 2012 at 02:14:12PM +0000, Jan Beulich wrote:
> >>> On 20.02.12 at 19:32, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > And only on Intel.. and only if it is a 32-bit dom0. If I do the same test 
> > with a 64-bit dom0 I do not see this problem.
> > 
> > Any thoughts of what it might be or what I should try out? My thought
> > was to swap out the hypervisor (use a Xen 4.0) or unstable and see if I get 
> > the same result.
> > But maybe there is something obvious out there?
> 
> Did you check that it's not the kernel image itself on that particular box
> that's corrupted?

The MD5sum looks to same. I think Ian's thoughts of looking at the different
libraries is my next thing to dive into.

Thanks!
> 
> Jan