[BUG] xenstored crash [xen-4.1.3] - likely tdb related

[Philipp Hahn <hahn@univention.de>]

Hello,

we now observed several xenstored crashes. After enabling writing core
filed I was able to capture the following stack trace through gdb:

> 0  talloc_chunk_from_ptr (ptr=0xff0000000000) at talloc.c:116
> 116             if ((tc->flags & ~0xF) != TALLOC_MAGIC) { 
> warning: not using untrusted file "/root/xen-4.1-4.1.3/xen-4.1.3/tools/xenstore/.gdbinit"
> (gdb) bt
> #0  talloc_chunk_from_ptr (ptr=0xff0000000000) at talloc.c:116
> #1  0x0000000000407edf in talloc_free (ptr=0xff0000000000) at talloc.c:551
> #2  0x000000000040a348 in tdb_open_ex (name=0x167d620 "/var/lib/xenstored/tdb.0x16a48b0", 
>     hash_size=<value optimized out>, tdb_flags=0, open_flags=<value optimized out>, mode=<value optimized out>, 
>     log_fn=0x4093b0 <null_log_fn>, hash_fn=<value optimized out>) at tdb.c:1958
> #3  0x000000000040a684 in tdb_open (name=0xff0000000000 <Address 0xff0000000000 out of bounds>, hash_size=0, 
>     tdb_flags=4254928, open_flags=-1, mode=3974450184) at tdb.c:1773
> #4  0x000000000040a70b in tdb_copy (tdb=0x16c9040, outfile=0x167d620 "/var/lib/xenstored/tdb.0x16a48b0")
>     at tdb.c:2124
> #5  0x0000000000406c2d in do_transaction_start (conn=0x167e310, in=<value optimized out>)
>     at xenstored_transaction.c:164
> #6  0x00000000004045ca in process_message (conn=0x167e310) at xenstored_core.c:1214
> #7  consider_message (conn=0x167e310) at xenstored_core.c:1261
> #8  handle_input (conn=0x167e310) at xenstored_core.c:1308
> #9  0x0000000000405170 in main (argc=<value optimized out>, argv=<value optimized out>) at xenstored_core.c:1964
> 
> (gdb) frame 2
> #2  0x000000000040a348 in tdb_open_ex (name=0x167d620 "/var/lib/xenstored/tdb.0x16a48b0", 
>     hash_size=<value optimized out>, tdb_flags=0, open_flags=<value optimized out>, mode=<value optimized out>, 
>     log_fn=0x4093b0 <null_log_fn>, hash_fn=<value optimized out>) at tdb.c:1958
> 1958            SAFE_FREE(tdb->locked);
> (gdb) print tdb->locked
> $3 = (struct tdb_lock_type *) 0xff0000000000

The "tdb->locked" address looks bogus.
I had a look at xen/tools/xenstore/tdb.c myself but did not spot any
obvious errors. As tdb_copy() looks like some internal function of tdb
and tdb has come from the SAMBA project, this looks more like a bug in
tdb then in xenstored.
I compared tdb between RELEASE-4.1.3 and master and didn't see any
interesting changes, so I'm not convinced that an update to 4.1.6 or
newer xen-4.x would solve this specific issue.
The crash is very annoying as the domains can no longer be managed or
migrated. As xenstored (AFAIK) can't be restarted, we currently have to
reboot the host to get the system back to a workable state.

Has someone seen that bug elsewhere?

Sincerely
Philipp
-- 
Philipp Hahn
Open Source Software Engineer

Univention GmbH
be open.
Mary-Somerville-Str. 1
D-28359 Bremen
Tel.: +49 421 22232-0
Fax : +49 421 22232-99
hahn@univention.de

http://www.univention.de/
Geschäftsführer: Peter H. Ganten
HRB 20755 Amtsgericht Bremen
Steuer-Nr.: 71-597-02876