All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jan Kiszka <jan.kiszka@web.de>
To: Peter Lieven <pl@dlh.net>
Cc: qemu-devel <qemu-devel@nongnu.org>, kvm@vger.kernel.org
Subject: Re: segmentation fault in qemu-kvm-0.14.0
Date: Wed, 09 Mar 2011 11:20:05 +0100	[thread overview]
Message-ID: <4D775455.10607@web.de> (raw)
In-Reply-To: <E319FC8C-43CE-4509-999B-0E5648FD1FA1@dlh.net>

[-- Attachment #1: Type: text/plain, Size: 2069 bytes --]

On 2011-03-09 11:16, Peter Lieven wrote:
> 
> Am 09.03.2011 um 08:37 schrieb Jan Kiszka:
> 
>> On 2011-03-08 23:53, Peter Lieven wrote:
>>> Hi,
>>>
>>> during testing of qemu-kvm-0.14.0 i can reproduce the following segfault. i have seen similar crash already in 0.13.0, but had no time to debug.
>>> my guess is that this segfault is related to the threaded vnc server which was introduced in qemu 0.13.0. the bug is only triggerable if a vnc
>>> client is attached. it might also be connected to a resolution change in the guest. i have a backtrace attached. the debugger is still running if someone
>>> needs more output
>>>
>>
>> ...
>>
>>> Thread 1 (Thread 0x7ffff7ff0700 (LWP 29038)):
>>> #0  0x0000000000000000 in ?? ()
>>> No symbol table info available.
>>> #1  0x000000000041d669 in main_loop_wait (nonblocking=0)
>>>    at /usr/src/qemu-kvm-0.14.0/vl.c:1388
>>
>> So we are calling a IOHandlerRecord::fd_write handler that is NULL.
>> Looking at qemu_set_fd_handler2, this may happen if that function is
>> called for an existing io-handler entry with non-NULL write handler,
>> passing a NULL write and a non-NULL read handler. And all this without
>> the global mutex held.
>>
>> And there are actually calls in vnc_client_write_plain and
>> vnc_client_write_locked (in contrast to vnc_write) that may generate
>> this pattern. It's probably worth validating that the iothread lock is
>> always held when qemu_set_fd_handler2 is invoked to confirm this race
>> theory, adding something like
>>
>> assert(pthread_mutex_trylock(&qemu_mutex) != 0);
>> (that's for qemu-kvm only)
> 
> qemu_mutex doesn't exists (anymore). i can only find qemu_global_mutex and qemu_fair_mutex.

In qemu-kvm, qemu-kvm.c contains that lock. In upstream, it's
qemu_global_mutex, but you will have to enable --enable-io-thread and
export it.

> 
> i try with qemu_global_mutex now. if thats not correct please tell.

I think we knokw know the call path, so you could also wait for Corentin
to send a patch and test that one.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

WARNING: multiple messages have this Message-ID (diff)
From: Jan Kiszka <jan.kiszka@web.de>
To: Peter Lieven <pl@dlh.net>
Cc: qemu-devel <qemu-devel@nongnu.org>, kvm@vger.kernel.org
Subject: [Qemu-devel] Re: segmentation fault in qemu-kvm-0.14.0
Date: Wed, 09 Mar 2011 11:20:05 +0100	[thread overview]
Message-ID: <4D775455.10607@web.de> (raw)
In-Reply-To: <E319FC8C-43CE-4509-999B-0E5648FD1FA1@dlh.net>

[-- Attachment #1: Type: text/plain, Size: 2069 bytes --]

On 2011-03-09 11:16, Peter Lieven wrote:
> 
> Am 09.03.2011 um 08:37 schrieb Jan Kiszka:
> 
>> On 2011-03-08 23:53, Peter Lieven wrote:
>>> Hi,
>>>
>>> during testing of qemu-kvm-0.14.0 i can reproduce the following segfault. i have seen similar crash already in 0.13.0, but had no time to debug.
>>> my guess is that this segfault is related to the threaded vnc server which was introduced in qemu 0.13.0. the bug is only triggerable if a vnc
>>> client is attached. it might also be connected to a resolution change in the guest. i have a backtrace attached. the debugger is still running if someone
>>> needs more output
>>>
>>
>> ...
>>
>>> Thread 1 (Thread 0x7ffff7ff0700 (LWP 29038)):
>>> #0  0x0000000000000000 in ?? ()
>>> No symbol table info available.
>>> #1  0x000000000041d669 in main_loop_wait (nonblocking=0)
>>>    at /usr/src/qemu-kvm-0.14.0/vl.c:1388
>>
>> So we are calling a IOHandlerRecord::fd_write handler that is NULL.
>> Looking at qemu_set_fd_handler2, this may happen if that function is
>> called for an existing io-handler entry with non-NULL write handler,
>> passing a NULL write and a non-NULL read handler. And all this without
>> the global mutex held.
>>
>> And there are actually calls in vnc_client_write_plain and
>> vnc_client_write_locked (in contrast to vnc_write) that may generate
>> this pattern. It's probably worth validating that the iothread lock is
>> always held when qemu_set_fd_handler2 is invoked to confirm this race
>> theory, adding something like
>>
>> assert(pthread_mutex_trylock(&qemu_mutex) != 0);
>> (that's for qemu-kvm only)
> 
> qemu_mutex doesn't exists (anymore). i can only find qemu_global_mutex and qemu_fair_mutex.

In qemu-kvm, qemu-kvm.c contains that lock. In upstream, it's
qemu_global_mutex, but you will have to enable --enable-io-thread and
export it.

> 
> i try with qemu_global_mutex now. if thats not correct please tell.

I think we knokw know the call path, so you could also wait for Corentin
to send a patch and test that one.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

  reply	other threads:[~2011-03-09 10:20 UTC|newest]

Thread overview: 90+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-03-08 22:53 segmentation fault in qemu-kvm-0.14.0 Peter Lieven
2011-03-08 22:53 ` [Qemu-devel] " Peter Lieven
2011-03-09  7:13 ` Corentin Chary
2011-03-09  7:26 ` Stefan Weil
2011-03-09  7:26   ` Stefan Weil
2011-03-09  7:39   ` Michael Tokarev
2011-03-09  7:39     ` Michael Tokarev
2011-03-09  9:22     ` Stefan Weil
2011-03-09  9:22       ` Stefan Weil
2011-03-09 10:00   ` Peter Lieven
2011-03-09 10:00     ` Peter Lieven
2011-03-15 12:53   ` Peter Lieven
2011-03-15 12:53     ` Peter Lieven
2011-03-15 18:52     ` Stefan Weil
2011-03-15 18:52       ` Stefan Weil
2011-03-09  7:37 ` Jan Kiszka
2011-03-09  7:37   ` [Qemu-devel] " Jan Kiszka
2011-03-09  8:50   ` Corentin Chary
2011-03-09  9:04     ` Jan Kiszka
2011-03-09  9:54       ` Corentin Chary
2011-03-09  9:58         ` Jan Kiszka
2011-03-09 10:02           ` Jan Kiszka
2011-03-09 10:06             ` Corentin Chary
2011-03-09 10:12               ` Jan Kiszka
2011-03-09 10:14                 ` Corentin Chary
2011-03-09 10:17                   ` Jan Kiszka
2011-03-09 10:41                     ` [PATCH] vnc: threaded server depends on io-thread Corentin Chary
2011-03-09 10:41                       ` [Qemu-devel] " Corentin Chary
2011-03-09 10:50                       ` Peter Lieven
2011-03-09 10:50                         ` [Qemu-devel] " Peter Lieven
2011-03-09 10:57                         ` Corentin Chary
2011-03-09 10:57                           ` [Qemu-devel] " Corentin Chary
2011-03-09 11:05                           ` Stefan Hajnoczi
2011-03-09 11:05                             ` [Qemu-devel] " Stefan Hajnoczi
2011-03-09 11:25                             ` Jan Kiszka
2011-03-09 11:25                               ` [Qemu-devel] " Jan Kiszka
2011-03-09 11:32                               ` Peter Lieven
2011-03-09 11:32                                 ` [Qemu-devel] " Peter Lieven
2011-03-09 11:33                                 ` Jan Kiszka
2011-03-09 11:33                                   ` [Qemu-devel] " Jan Kiszka
2011-03-09 11:42                       ` Jan Kiszka
2011-03-09 11:42                         ` [Qemu-devel] " Jan Kiszka
2011-03-09 12:50                         ` Peter Lieven
2011-03-09 12:50                           ` [Qemu-devel] " Peter Lieven
2011-03-09 13:21                         ` [PATCH v2] " Corentin Chary
2011-03-09 13:21                           ` [Qemu-devel] " Corentin Chary
2011-03-09 13:42                           ` Corentin Chary
2011-03-09 13:42                             ` [Qemu-devel] " Corentin Chary
2011-03-09 13:51                           ` Paolo Bonzini
2011-03-09 13:51                             ` [Qemu-devel] " Paolo Bonzini
2011-03-09 13:59                             ` Corentin Chary
2011-03-09 13:59                               ` [Qemu-devel] " Corentin Chary
2011-03-10 12:59                         ` [PATCH 1/2] sockets: add qemu_socketpair() Corentin Chary
2011-03-10 12:59                           ` [Qemu-devel] " Corentin Chary
2011-03-10 12:59                         ` [PATCH 2/2] vnc: don't mess up with iohandlers in the vnc thread Corentin Chary
2011-03-10 12:59                           ` [Qemu-devel] " Corentin Chary
2011-03-10 13:06                           ` Paolo Bonzini
2011-03-10 13:06                             ` [Qemu-devel] " Paolo Bonzini
2011-03-10 13:45                             ` Anthony Liguori
2011-03-10 13:45                               ` [Qemu-devel] " Anthony Liguori
2011-03-10 13:54                               ` Corentin Chary
2011-03-10 13:54                                 ` [Qemu-devel] " Corentin Chary
2011-03-10 13:58                                 ` Paolo Bonzini
2011-03-10 13:58                                   ` [Qemu-devel] " Paolo Bonzini
2011-03-10 13:56                               ` Paolo Bonzini
2011-03-10 13:56                                 ` [Qemu-devel] " Paolo Bonzini
2011-03-10 13:47                           ` Peter Lieven
2011-03-10 13:47                             ` [Qemu-devel] " Peter Lieven
2011-03-10 15:13                         ` [PATCH v5] " Corentin Chary
2011-03-10 15:13                           ` [Qemu-devel] " Corentin Chary
2011-03-14  9:19                           ` Corentin Chary
2011-03-14  9:19                             ` [Qemu-devel] " Corentin Chary
2011-03-14  9:55                             ` Peter Lieven
2011-03-14  9:55                               ` [Qemu-devel] " Peter Lieven
2011-03-15 16:55                             ` Peter Lieven
2011-03-15 16:55                               ` [Qemu-devel] " Peter Lieven
2011-03-15 18:07                               ` Peter Lieven
2011-03-15 18:07                                 ` [Qemu-devel] " Peter Lieven
2011-03-09 10:02   ` segmentation fault in qemu-kvm-0.14.0 Peter Lieven
2011-03-09 10:02     ` [Qemu-devel] " Peter Lieven
2011-03-09 10:16   ` Peter Lieven
2011-03-09 10:16     ` [Qemu-devel] " Peter Lieven
2011-03-09 10:20     ` Jan Kiszka [this message]
2011-03-09 10:20       ` Jan Kiszka
2011-03-09 10:31       ` Peter Lieven
2011-03-09 10:31         ` [Qemu-devel] " Peter Lieven
2011-03-09 11:20   ` Paolo Bonzini
2011-03-09 11:20     ` [Qemu-devel] " Paolo Bonzini
2011-03-09 11:44     ` Jan Kiszka
2011-03-09 11:44       ` [Qemu-devel] " Jan Kiszka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4D775455.10607@web.de \
    --to=jan.kiszka@web.de \
    --cc=kvm@vger.kernel.org \
    --cc=pl@dlh.net \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.