From mboxrd@z Thu Jan  1 00:00:00 1970
From: Peter Lieven <pl@dlhnet.de>
Subject: Re: qemu-kvm-1.0 crashes with threaded vnc server?
Date: Thu, 28 Jun 2012 12:21:11 +0200
Message-ID: <4FEC3017.2050301@dlhnet.de>
References: <4F340B8A.2020907@dlh.net>	<CAHR064jArbRzyCC23yejFCvSn_uwkHs50AiAhQCG8UCi2LxhwA@mail.gmail.com>	<4F5F2F82.3040000@dlh.net>	<CAHR064gRmvaQPHzFUwdk_wv8veNY6A2QkLmLj+po1RVxjHzDKQ@mail.gmail.com>	<E89A1874-4942-4CFC-A407-71B903D55ABA@suse.de>	<4FE9CEF5.8030601@dlhnet.de>
	<CAHR064i=6xaWckXXuZwZuzFf2OsR9ixnWVJfdOu1w6DPeNfmEg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: multipart/alternative;
	boundary="------------020903080102070606010603"
Cc: kvm@vger.kernel.org, Peter Lieven <pl@dlh.net>,
	Qemu-development List <qemu-devel@nongnu.org>,
	Alexander Graf <agraf@suse.de>, weil@mail.berlios.de, anthony@codemonkey.ws
To: Corentin Chary <corentin.chary@gmail.com>
Return-path: <qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org>
In-Reply-To: <CAHR064i=6xaWckXXuZwZuzFf2OsR9ixnWVJfdOu1w6DPeNfmEg@mail.gmail.com>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org
Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org
List-Id: kvm.vger.kernel.org

This is a multi-part message in MIME format.
--------------020903080102070606010603
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable

On 28.06.2012 12:18, Corentin Chary wrote:
>
> Please use "gdb -p <pid>" to attach qemu when stuck and use it to dump=20
> a full backtrace. Note that I never tested qemu-kvm and the vnc code=20
> was originally not written for it, so the locking stuff may be broken=20
> if the codebase differs from upstream qemu too much.
>
Hi Corentin,

it seems that this issue is not caused by the threaded vnc server. The=20
race condition is also reproducible if the threaded vnc server is=20
deactivated.

Sorry for the noise.

Peter

> Le 26 juin 2012 17:02, "Peter Lieven" <pl@dlhnet.de=20
> <mailto:pl@dlhnet.de>> a =C3=A9crit :
>
>     On 13.03.2012 16:06, Alexander Graf wrote:
>
>         On 13.03.2012, at 16:05, Corentin Chary wrote:
>
>             On Tue, Mar 13, 2012 at 12:29 PM, Peter Lieven<pl@dlh.net
>             <mailto:pl@dlh.net>>  wrote:
>
>                 On 11.02.2012 09:55, Corentin Chary wrote:
>
>                     On Thu, Feb 9, 2012 at 7:08 PM, Peter
>                     Lieven<pl@dlh.net <mailto:pl@dlh.net>>   wrote:
>
>                         Hi,
>
>                         is anyone aware if there are still problems
>                         when enabling the threaded
>                         vnc
>                         server?
>                         I saw some VMs crashing when using a qemu-kvm
>                         build with
>                         --enable-vnc-thread.
>
>                         qemu-kvm-1.0[22646]: segfault at 0 ip
>                         00007fec1ca7ea0b sp
>                         00007fec19d056d0
>                         error 6 in libz.so.1.2.3.3[7fec1ca75000+16000]
>                         qemu-kvm-1.0[26056]: segfault at 7f06d8d6e010
>                         ip 00007f06e0a30d71 sp
>                         00007f06df035748 error 6 in libc-2.11.1.so
>                         <http://libc-2.11.1.so>[7f06e09aa000+17a000]
>
>                         I had no time to debug further. It seems to
>                         happen shortly after
>                         migrating,
>                         but thats uncertain. At least the segfault in
>                         libz seems to
>                         give a hint to VNC since I cannot image of any
>                         other part of qemu-kvm
>                         using
>                         libz except for VNC server.
>
>                         Thanks,
>                         Peter
>
>
>                     Hi Peter,
>                     I found two patches on my git tree that I sent
>                     long ago but somehow
>                     get lost on the mailing list. I rebased the tree
>                     but did not have the
>                     time (yet) to test them.
>                     http://git.iksaif.net/?p=3Dqemu.git;a=3Dshortlog;h=3D=
refs/heads/wip
>                     Feel free to try them. If QEMU segfault again,
>                     please send a full gdb
>                     backtrace / valgrind trace / way to reproduce :).
>                     Thanks,
>
>                 I have seen no more crashes with these to patches
>                 applied. I would suggest
>                 it would be good to push them to the master repository.
>
>                 Thank you,
>                 Peter
>
>             Ccing Alexander,
>
>         Ah, cool. Corentin, I think you're right now the closest thing
>         we have to a maintainer for VNC. Could you please just send
>         out a pull request for those?
>
>     hi all,
>
>     i suspect there is still a problem with the threaded vnc server.
>     its just a guess, but we saw a resonable number of vms hanging in t=
he
>     last weeks. hanging meaning the emulation is stopped and the
>     qemu-kvm process does no longer react, not on monitor, not on vnc,
>     not on qmp.
>     why i suspect the threaded vnc server is that in all cases we have
>     analyzed this happened with an open vnc session and only on nodes
>     with the threaded vnc server
>     enabled. it might also be the case that this happens at a
>     resolution change. is there anything known or has someone an idea?
>
>     we are running qemu-kvm 1.0.1 with
>
>      vnc: don't mess up with iohandlers in the vnc thread
>
>      vnc: Limit r/w access to size of allocated memory
>
>     compiled in.
>
>     unfortunately, i was not yet able to reproduce this with a
>     debugger attached.
>
>     thanks,
>     peter
>
>
>         Alex
>
>


--------------020903080102070606010603
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <meta content=3D"text/html; charset=3DUTF-8" http-equiv=3D"Content-Ty=
pe">
  </head>
  <body bgcolor=3D"#ffffff" text=3D"#000000">
    On 28.06.2012 12:18, Corentin Chary wrote:
    <blockquote
cite=3D"mid:CAHR064i=3D6xaWckXXuZwZuzFf2OsR9ixnWVJfdOu1w6DPeNfmEg@mail.gm=
ail.com"
      type=3D"cite">
      <p>Please use "gdb -p &lt;pid&gt;" to attach qemu when stuck and
        use it to dump a full backtrace. Note that I never tested
        qemu-kvm and the vnc code was originally not written for it, so
        the locking stuff may be broken if the codebase differs from
        upstream qemu too much.</p>
    </blockquote>
    Hi Corentin,<br>
    <br>
    it seems that this issue is not caused by the threaded vnc server.
    The race condition is also reproducible if the threaded vnc server
    is deactivated.<br>
    <br>
    Sorry for the noise.<br>
    <br>
    Peter<br>
    <br>
    <blockquote
cite=3D"mid:CAHR064i=3D6xaWckXXuZwZuzFf2OsR9ixnWVJfdOu1w6DPeNfmEg@mail.gm=
ail.com"
      type=3D"cite">
      <div class=3D"gmail_quote">Le 26 juin 2012 17:02, "Peter Lieven"
        &lt;<a moz-do-not-send=3D"true" href=3D"mailto:pl@dlhnet.de">pl@d=
lhnet.de</a>&gt;
        a =C3=A9crit=C2=A0:<br type=3D"attribution">
        <blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt
          0.8ex; border-left: 1px solid rgb(204, 204, 204);
          padding-left: 1ex;">
          On 13.03.2012 16:06, Alexander Graf wrote:<br>
          <blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt
            0.8ex; border-left: 1px solid rgb(204, 204, 204);
            padding-left: 1ex;">
            On 13.03.2012, at 16:05, Corentin Chary wrote:<br>
            <br>
            <blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0p=
t
              0.8ex; border-left: 1px solid rgb(204, 204, 204);
              padding-left: 1ex;">
              On Tue, Mar 13, 2012 at 12:29 PM, Peter Lieven&lt;<a
                moz-do-not-send=3D"true" href=3D"mailto:pl@dlh.net"
                target=3D"_blank">pl@dlh.net</a>&gt; =C2=A0wrote:<br>
              <blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt =
0pt
                0.8ex; border-left: 1px solid rgb(204, 204, 204);
                padding-left: 1ex;">
                On 11.02.2012 09:55, Corentin Chary wrote:<br>
                <blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0p=
t
                  0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204);
                  padding-left: 1ex;">
                  On Thu, Feb 9, 2012 at 7:08 PM, Peter Lieven&lt;<a
                    moz-do-not-send=3D"true" href=3D"mailto:pl@dlh.net"
                    target=3D"_blank">pl@dlh.net</a>&gt; =C2=A0 wrote:<br=
>
                  <blockquote class=3D"gmail_quote" style=3D"margin: 0pt =
0pt
                    0pt 0.8ex; border-left: 1px solid rgb(204, 204,
                    204); padding-left: 1ex;">
                    Hi,<br>
                    <br>
                    is anyone aware if there are still problems when
                    enabling the threaded<br>
                    vnc<br>
                    server?<br>
                    I saw some VMs crashing when using a qemu-kvm build
                    with<br>
                    --enable-vnc-thread.<br>
                    <br>
                    qemu-kvm-1.0[22646]: segfault at 0 ip
                    00007fec1ca7ea0b sp<br>
                    00007fec19d056d0<br>
                    error 6 in libz.so.1.2.3.3[7fec1ca75000+16000]<br>
                    qemu-kvm-1.0[26056]: segfault at 7f06d8d6e010 ip
                    00007f06e0a30d71 sp<br>
                    00007f06df035748 error 6 in <a
                      moz-do-not-send=3D"true"
                      href=3D"http://libc-2.11.1.so" target=3D"_blank">li=
bc-2.11.1.so</a>[7f06e09aa000+17a000]<br>
                    <br>
                    I had no time to debug further. It seems to happen
                    shortly after<br>
                    migrating,<br>
                    but thats uncertain. At least the segfault in libz
                    seems to<br>
                    give a hint to VNC since I cannot image of any other
                    part of qemu-kvm<br>
                    using<br>
                    libz except for VNC server.<br>
                    <br>
                    Thanks,<br>
                    Peter<br>
                    <br>
                    <br>
                  </blockquote>
                  Hi Peter,<br>
                  I found two patches on my git tree that I sent long
                  ago but somehow<br>
                  get lost on the mailing list. I rebased the tree but
                  did not have the<br>
                  time (yet) to test them.<br>
                  <a moz-do-not-send=3D"true"
                    href=3D"http://git.iksaif.net/?p=3Dqemu.git;a=3Dshort=
log;h=3Drefs/heads/wip"
                    target=3D"_blank">http://git.iksaif.net/?p=3Dqemu.git=
;a=3Dshortlog;h=3Drefs/heads/wip</a><br>
                  Feel free to try them. If QEMU segfault again, please
                  send a full gdb<br>
                  backtrace / valgrind trace / way to reproduce :).<br>
                  Thanks,<br>
                  <br>
                </blockquote>
                I have seen no more crashes with these to patches
                applied. I would suggest<br>
                it would be good to push them to the master repository.<b=
r>
                <br>
                Thank you,<br>
                Peter<br>
                <br>
              </blockquote>
              Ccing Alexander,<br>
            </blockquote>
            Ah, cool. Corentin, I think you're right now the closest
            thing we have to a maintainer for VNC. Could you please just
            send out a pull request for those?<br>
          </blockquote>
          hi all,<br>
          <br>
          i suspect there is still a problem with the threaded vnc
          server. its just a guess, but we saw a resonable number of vms
          hanging in the<br>
          last weeks. hanging meaning the emulation is stopped and the
          qemu-kvm process does no longer react, not on monitor, not on
          vnc, not on qmp.<br>
          why i suspect the threaded vnc server is that in all cases we
          have analyzed this happened with an open vnc session and only
          on nodes with the threaded vnc server<br>
          enabled. it might also be the case that this happens at a
          resolution change. is there anything known or has someone an
          idea?<br>
          <br>
          we are running qemu-kvm 1.0.1 with<br>
          <br>
          =C2=A0vnc: don't mess up with iohandlers in the vnc thread<br>
          <br>
          =C2=A0vnc: Limit r/w access to size of allocated memory<br>
          <br>
          compiled in.<br>
          <br>
          unfortunately, i was not yet able to reproduce this with a
          debugger attached.<br>
          <br>
          thanks,<br>
          peter<br>
          <br>
          <blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt
            0.8ex; border-left: 1px solid rgb(204, 204, 204);
            padding-left: 1ex;">
            <br>
            Alex<br>
            <br>
          </blockquote>
          <br>
        </blockquote>
      </div>
    </blockquote>
    <br>
  </body>
</html>

--------------020903080102070606010603--