From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adEWz-0008OC-5l for qemu-devel@nongnu.org; Tue, 08 Mar 2016 05:06:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1adEWx-0000sj-Vn for qemu-devel@nongnu.org; Tue, 08 Mar 2016 05:06:21 -0500 Date: Tue, 8 Mar 2016 10:06:12 +0000 From: Stefan Hajnoczi Message-ID: <20160308100612.GC9407@stefanha-x1.localdomain> References: <56DD7414.9080306@de.ibm.com> <20160307170139.GB26074@stefanha-x1.localdomain> <56DDCFE1.4000808@de.ibm.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="f+W+jCU1fRNres8c" Content-Disposition: inline In-Reply-To: <56DDCFE1.4000808@de.ibm.com> Subject: Re: [Qemu-devel] strange crash in tracked_request_begin List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Christian Borntraeger Cc: Paolo Bonzini , qemu-devel , qemu-block@nongnu.org --f+W+jCU1fRNres8c Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 07, 2016 at 08:00:49PM +0100, Christian Borntraeger wrote: > On 03/07/2016 06:01 PM, Stefan Hajnoczi wrote: > > On Mon, Mar 07, 2016 at 01:29:08PM +0100, Christian Borntraeger wrote: > >> Folks, > >> > >> I had a crash of a qemu guest in tracked_request_begin. > >> The testcase was a guest with ramdisk/kernel that reboots in a=20 > >> loop. (about 10 times per second) with a single null-co disk=20 > >> attached. No idea how to reproduce this, seems to be a lucky hit. > >> > >> (gdb) bt > >> #0 0x00000000101db5ba in tracked_request_begin (req=3Dreq@entry=3D0x3= ff90f1bdc0, bs=3Dbs@entry=3D0x42a39190, offset=3Doffset@entry=3D0, bytes=3D= bytes@entry=3D4096, type=3Dtype@entry=3DBDRV_TRACKED_READ) > >> at /home/cborntra/REPOS/qemu/block/io.c:390 > >> #1 0x00000000101de91e in bdrv_co_do_preadv (bs=3D0x42a39190, offset= =3D0, bytes=3D4096, qiov=3D0x3ff7400cbd8, flags=3D, flags@en= try=3D(unknown: 0)) > >> at /home/cborntra/REPOS/qemu/block/io.c:1001 > >> #2 0x00000000101dfc3e in bdrv_co_do_readv (flags=3D(unknown: 0), qiov= =3D, nb_sectors=3D, sector_num=3D, bs=3D) > >> at /home/cborntra/REPOS/qemu/block/io.c:1024 > >> #3 bdrv_co_do_rw (opaque=3D0x3ff7400e370) at /home/cborntra/REPOS/qem= u/block/io.c:2173 > >> #4 0x000000001022d8f6 in coroutine_trampoline (i0=3D, = i1=3D-1946150928) at /home/cborntra/REPOS/qemu/util/coroutine-ucontext.c:79 > >> #5 0x000003ff95ed150a in __makecontext_ret () from /lib64/libc.so.6 > >> > >> looking at the code we are at > >> > >> QLIST_INSERT_HEAD(&bs->tracked_requests, req, list); > >> which translates to > >> > >> if (((req)->list.le_next =3D (&bs->tracked_requests)->lh_first) !=3D N= ULL)=20 > >> (&bs->tracked_requests)->lh_first->list.le_prev =3D &(req)->list.l= e_next; > >> (&bs->tracked_requests)->lh_first =3D (req); =20 > >> (req)->list.le_prev =3D &(&bs->tracked_requests)->lh_first; > >> > >> gdb says, that (&bs->tracked_requests)->lh_first) is zero in the coref= ile > >> (gdb) print /x bs->tracked_requests > >> $6 =3D {lh_first =3D 0x0} > >> > >> Now looking at the code I am asking myself if this can happen in paral= lel > >> to another code that touches tracked_requests, because gcc seems to re= ad > >> &bs->tracked_requests)->lh_first twice (first to check the value, then > >> to use it as pointer) > >=20 > > tracked_requests is protected by AioContext. Perhaps something is doing > > I/O without acquiring AioContext? >=20 > Hmm, the guest was rebooting, which resets all devices. Maybe something > in that code is still not right? I will have a look. virtio_blk_reset() does acquire AioContext so at least that part should be safe with running IOThreads. Stefan --f+W+jCU1fRNres8c Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJW3qQUAAoJEJykq7OBq3PIcZcIALA4lcX3jRAVkArgmL1TyO3F Sl7nPotaoU2zg5KoBV3Ecghegy7VqTGmln6804K/cA96BlfSNLq5w+/BLwX84Gyg SzGQeRtRohoPTbR81Yf6YRgq26WUc48uhnj/Ga5GRl4lsXwQ+xKYtKf7VbmIFFDl C4bJSQd3Yhgoo54wvJQEo3+M1IqG5PThv7Ri2ZlavGHuStAh7tywvReJ4aamXm2x 0uu11I1AccmppMeMV2xLWrmleTILwuBi+R1kaBg7JcaqvwwZJ40Si5oVCIeozhTA ZVcUoCZ714HKUhhDCM0wmH+vXFJUIUgDo+b4B4FOo5z9+IVjWklxcfgAHYgans8= =UHJQ -----END PGP SIGNATURE----- --f+W+jCU1fRNres8c--