From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43497) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1boogY-00022j-JT for qemu-devel@nongnu.org; Tue, 27 Sep 2016 05:28:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1boogT-0003Uz-FE for qemu-devel@nongnu.org; Tue, 27 Sep 2016 05:28:21 -0400 Received: from mail-lf0-f51.google.com ([209.85.215.51]:33959) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1boogT-0003UZ-3s for qemu-devel@nongnu.org; Tue, 27 Sep 2016 05:28:17 -0400 Received: by mail-lf0-f51.google.com with SMTP id y6so15895794lff.1 for ; Tue, 27 Sep 2016 02:28:16 -0700 (PDT) Date: Tue, 27 Sep 2016 10:27:12 +0100 From: Stefan Hajnoczi Message-ID: <20160927092712.GA563@stefanha-x1.localdomain> References: <03BF752A-0E6A-4AAD-A310-DFACDF0B8339@nutanix.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline In-Reply-To: <03BF752A-0E6A-4AAD-A310-DFACDF0B8339@nutanix.com> Subject: Re: [Qemu-devel] Live migration without bdrv_drain_all() List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Felipe Franciosi Cc: qemu-devel , Mike Cui , Kevin Wolf , Paolo Bonzini , Juan Quintela , "Dr. David Alan Gilbert" --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 29, 2016 at 06:56:42PM +0000, Felipe Franciosi wrote: > Heya! >=20 > > On 29 Aug 2016, at 08:06, Stefan Hajnoczi wrote: > >=20 > > At KVM Forum an interesting idea was proposed to avoid > > bdrv_drain_all() during live migration. Mike Cui and Felipe Franciosi > > mentioned running at queue depth 1. It needs more thought to make it > > workable but I want to capture it here for discussion and to archive > > it. > >=20 > > bdrv_drain_all() is synchronous and can cause VM downtime if I/O > > requests hang. We should find a better way of quiescing I/O that is > > not synchronous. Up until now I thought we should simply add a > > timeout to bdrv_drain_all() so it can at least fail (and live > > migration would fail) if I/O is stuck instead of hanging the VM. But > > the following approach is also interesting... > >=20 > > During the iteration phase of live migration we could limit the queue > > depth so points with no I/O requests in-flight are identified. At > > these points the migration algorithm has the opportunity to move to > > the next phase without requiring bdrv_drain_all() since no requests > > are pending. >=20 > I actually think that this "io quiesced state" is highly unlikely to _jus= t_ happen on a busy guest. The main idea behind running at QD1 is to natura= lly throttle the guest and make it easier to "force quiesce" the VQs. >=20 > In other words, if the guest is busy and we run at QD1, I would expect th= e rings to be quite full of pending (ie. unprocessed) requests. At the same= time, I would expect that a call to bdrv_drain_all() (as part of do_vm_sto= p()) should complete much quicker. >=20 > Nevertheless, you mentioned that this is still problematic as that single= outstanding IO could block, leaving the VM paused for longer. >=20 > My suggestion is therefore that we leave the vCPUs running, but stop pick= ing up requests from the VQs. Provided nothing blocks, you should reach the= "io quiesced state" fairly quickly. If you don't, then the VM is at least = still running (despite seeing no progress on its VQs). >=20 > Thoughts on that? If the guest experiences a hung disk it may enter error recovery. QEMU should avoid this so the guest doesn't remount file systems read-only. This can be solved by only quiescing the disk for, say, 30 seconds at a time. If we don't reach a point where live migration can proceed during those 30 seconds then the disk will service requests again temporarily to avoid upsetting the guest. I wonder if Juan or David have any thoughts from the live migration perspective? Stefan --9jxsPFA5p3P2qPhR Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJX6jtwAAoJEJykq7OBq3PI43cH/iO5TapN7AimUqm72wlFmgPS lQDkVZNiGS2oXrepi6tPkefFUipdkL2j1P6yDhaRHiwbzRAWEBm9lDWfB2R6nuJo 7m+XSfyvJMT0oU52WFQCVwTTuOGS+qv2RtOsLZFEh2FAdVHvZwa4SJGZ4TYThlYr f/SvpVHOZ5lnJZxgzd5yvcsvsh6GG99fRBDIDXUYb8314rC+rskH2T27dq/ZOtTM sMwERHq/Mh17KP19zkxnljmEpkrkAoYq/OxWbFOoDShrtOpad00YfYgZU2yRtgyj FXGzSxIVxp6EprBhnUTetZzESb2c4G2dBEBrOGZnwSs+ERGK2W/QYAfbSnpSx9g= =owyK -----END PGP SIGNATURE----- --9jxsPFA5p3P2qPhR--