Memory corruption when submitting a huge number of SMS

Memory corruption when submitting a huge number of SMS

[Holger Hans Peter Freyther]

[-- Attachment #1: Type: text/plain, Size: 4021 bytes --]

Hi,

I am using our modem bank with four SIM cards and send 1000 per
SIM card and ofono is segfaulting in the process of doing so. The
crash appears to be due a memory corruption and I wonder if you
have experience in debugging those. This is ofono-1.11 with some
patches to build on debian stable.

kind regards
	holger


gdb backtrace:
#0  0xb7e51c02 in dbus_connection_get_dispatch_status ()
   from /lib/i386-linux-gnu/libdbus-1.so.3
#1  0x08054be6 in watch_func (chan=0x8712e78, cond=0, data=0x8713590) at gdbus/mainloop.c:105
#2  0xb7f1b6fe in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
#3  0xb7eda6d3 in g_main_context_dispatch () from /lib/i386-linux-gnu/libglib-2.0.so.0
#4  0xb7edaa70 in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
#5  0xb7edaecb in g_main_loop_run () from /lib/i386-linux-gnu/libglib-2.0.so.0
#6  0x080d378a in main (argc=1, argv=0xbffff764) at src/main.c:247


valgrind:
==10495== Invalid read of size 4
==10495==    at 0x8054BDB: watch_func (mainloop.c:105)
==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x4208E45: (below main) (libc-start.c:228)
==10495==  Address 0x47ba830 is 8 bytes inside a block of size 12 free'd
==10495==    at 0x402768C: free (vg_replace_malloc.c:446)
==10495==    by 0x40D177A: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x41A81AC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x418AD21: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x41A6BEC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x41A7901: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x41A6028: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x418DA0A: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495== 
==10495== Invalid read of size 4
==10495==    at 0x8054BE6: watch_func (mainloop.c:106)
==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x4208E45: (below main) (libc-start.c:228)
==10495==  Address 0x47ba830 is 8 bytes inside a block of size 12 free'd
==10495==    at 0x402768C: free (vg_replace_malloc.c:446)
==10495==    by 0x40D177A: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x41A81AC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x418AD21: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x41A6BEC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x41A7901: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x41A6028: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x418DA0A: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
==10495== 

Re: Memory corruption when submitting a huge number of SMS

[Denis Kenzior]

[-- Attachment #1: Type: text/plain, Size: 4132 bytes --]

Hi Holger,

> gdb backtrace:
> #0  0xb7e51c02 in dbus_connection_get_dispatch_status ()
>     from /lib/i386-linux-gnu/libdbus-1.so.3
> #1  0x08054be6 in watch_func (chan=0x8712e78, cond=0, data=0x8713590) at gdbus/mainloop.c:105
> #2  0xb7f1b6fe in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
> #3  0xb7eda6d3 in g_main_context_dispatch () from /lib/i386-linux-gnu/libglib-2.0.so.0
> #4  0xb7edaa70 in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
> #5  0xb7edaecb in g_main_loop_run () from /lib/i386-linux-gnu/libglib-2.0.so.0
> #6  0x080d378a in main (argc=1, argv=0xbffff764) at src/main.c:247
>

Not a whole lot to go on.  Looks like the crash happens in gdbus.  Is 
the dbus daemon crashing or bumping ofono off the bus by any chance?

>
> valgrind:
> ==10495== Invalid read of size 4
> ==10495==    at 0x8054BDB: watch_func (mainloop.c:105)
> ==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x4208E45: (below main) (libc-start.c:228)
> ==10495==  Address 0x47ba830 is 8 bytes inside a block of size 12 free'd
> ==10495==    at 0x402768C: free (vg_replace_malloc.c:446)
> ==10495==    by 0x40D177A: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x41A81AC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x418AD21: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x41A6BEC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x41A7901: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x41A6028: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x418DA0A: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==
> ==10495== Invalid read of size 4
> ==10495==    at 0x8054BE6: watch_func (mainloop.c:106)
> ==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x4208E45: (below main) (libc-start.c:228)
> ==10495==  Address 0x47ba830 is 8 bytes inside a block of size 12 free'd
> ==10495==    at 0x402768C: free (vg_replace_malloc.c:446)
> ==10495==    by 0x40D177A: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x41A81AC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x418AD21: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x41A6BEC: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x41A7901: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x41A6028: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x418DA0A: ??? (in /lib/i386-linux-gnu/libdbus-1.so.3.7.2)
> ==10495==    by 0x410C6FD: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CB6D2: g_main_context_dispatch (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBA6F: ??? (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==    by 0x40CBECA: g_main_loop_run (in /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4)
> ==10495==

Have you tried making a debug build and seeing if you get slightly 
better debug information?  Just set CFLAGS to -O0 and --enable-debug 
when configuring.

Regards,
-Denis

Re: Memory corruption when submitting a huge number of SMS

[Holger Hans Peter Freyther]

[-- Attachment #1: Type: text/plain, Size: 694 bytes --]

On Mon, Nov 12, 2012 at 01:29:10PM -0600, Denis Kenzior wrote:
> Hi Holger,
> 
> Not a whole lot to go on.  Looks like the crash happens in gdbus.
> Is the dbus daemon crashing or bumping ofono off the bus by any
> chance?

the dbus-daemon is running since the start of my laptop.

> 
> Have you tried making a debug build and seeing if you get slightly
> better debug information?  Just set CFLAGS to -O0 and --enable-debug
> when configuring.

not yet. I might need to travel this week so it might take until Wed or Fri
to test this. Is ofono using multiple threads? this somehow sounds like a mismatch
of first deregister from the glib event loop and then free...

holger

Re: Memory corruption when submitting a huge number of SMS

[Denis Kenzior]

[-- Attachment #1: Type: text/plain, Size: 472 bytes --]

Hi Holger,

>> Have you tried making a debug build and seeing if you get slightly
>> better debug information?  Just set CFLAGS to -O0 and --enable-debug
>> when configuring.
>
> not yet. I might need to travel this week so it might take until Wed or Fri
> to test this. Is ofono using multiple threads? this somehow sounds like a mismatch
> of first deregister from the glib event loop and then free...
>

No, oFono doesn't use threads.

Regards,
-Denis