From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:42036) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TByhZ-00083q-7g for qemu-devel@nongnu.org; Wed, 12 Sep 2012 21:58:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TByhX-000582-Ls for qemu-devel@nongnu.org; Wed, 12 Sep 2012 21:58:45 -0400 Received: from e4.ny.us.ibm.com ([32.97.182.144]:44540) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TByhX-00057W-HM for qemu-devel@nongnu.org; Wed, 12 Sep 2012 21:58:43 -0400 Received: from /spool/local by e4.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 12 Sep 2012 21:58:42 -0400 Received: from d01relay03.pok.ibm.com (d01relay03.pok.ibm.com [9.56.227.235]) by d01dlp03.pok.ibm.com (Postfix) with ESMTP id CA989C9004A for ; Wed, 12 Sep 2012 21:58:39 -0400 (EDT) Received: from d01av04.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by d01relay03.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q8D1wdKY136880 for ; Wed, 12 Sep 2012 21:58:39 -0400 Received: from d01av04.pok.ibm.com (loopback [127.0.0.1]) by d01av04.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q8D1wdGU008171 for ; Wed, 12 Sep 2012 21:58:39 -0400 From: Anthony Liguori In-Reply-To: <20120912161206.GO26835@redhat.com> References: <1347451046-5513-1-git-send-email-lilei@linux.vnet.ibm.com> <20120912161206.GO26835@redhat.com> Date: Wed, 12 Sep 2012 20:58:00 -0500 Message-ID: <87vcfizntj.fsf@codemonkey.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: [Qemu-devel] [RFC v3 ATCH 0/5] char: expose CirMemCharDriver and provide QMP interface List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Daniel P. Berrange" , Lei Li Cc: lcapitulino@redhat.com, eblake@redhat.com, qemu-devel@nongnu.org, armbru@redhat.com "Daniel P. Berrange" writes: > On Wed, Sep 12, 2012 at 07:57:21PM +0800, Lei Li wrote: >> This RFC series attempts to convert the MemCharDriver to use a circular >> buffer for input and output, expose it to users by introducing QMP commands >> memchar_write and memchar_read and via the command line like the other >> CharDriverStates. >> >> Serial ports in qemu always use CharDriverStates as there backends, >> Right now, all of our backends always try to write the data from the >> guest to a socket or file. The concern from OpenStack is that this could >> lead to unbounded disk space usage since they log the serial output. >> For more detail of the background info: >> https://bugs.launchpad.net/nova/+bug/832507 > > Unbounded disk usage is only relevant if telling QEMU to write directly > to its file backend. If you use a socket backend the mgmt app can provide > whatever policy it desires. > >> So we want to use a circular buffer in QEMU instead, and then OpenStack >> can periodically read the buffer in QEMU and log it. > > With any circular buffer you obviously have a race condition where > the buffer may overflow before the application can consume the data. > By implementing the circular buffer in QEMU you are imposing a > specific policy for overflow on the applications using QEMU, namely > that data gets overwritten/lost. > > If the circular buffering is done outside QEMU, then the application > can implement a variety of policies on overflow. At the very least > they can detect when overflow would occur, and insert a marker to > the effect that there is a log discontinuity. Or they can pause the > VM for a period of time, or reduce its scheduling priority, or any > number of different things. > > The further advantage of doing it outside QEMU, is that OpenStack will > work with all existing QEMU/KVM/libvirt versions. I'm not sure what is the best solution for libvirt and OpenStack but I think you're missing a few key points. CDS doesn't propagate flow control to the guest (in some cases, it simply can't because hardware doesn't). That means that there has to be some policy in QEMU about what to do with data that cannot be written to a socket. Today we simply drop new data. This adds a mechanism where we drop old data. For some cases, dropping old data is much nicer than dropping new data. But there are other compelling use-cases for this beyond libvirt. This will allow us to implement a 'console' command which will be pretty nice within HMP. It also provides a very nice way to write tests. It's much easier to use something like this from qtest than it is to setup a socket in in listen mode and then queue incoming data to be read. Regards, Anthony Liguori > I think most of the discussion in the quoted OpenStack bug is rather > short sighted only focusing on the immediate needs for their current > 'get_log_output' API. I don't think this will be satisfactory for > OpenStack in the medium-long term. IMHO OpenStack needs to provide > a more comprensive logging capability than what is does today, and > thus this proposed QEMU feature would have a short lifetime of usefulness > to OpenStack. I've already recommended that OpenStack take advantage > of conserver to setup logging of all VMs, though there were some > issues around that. It is entirely possible for OpenStack to provide > its own logging system to process VM logs into fixed size files, as > well as satisfying the needs of its get_log_output API for circular > buffers. > > Regards, > Daniel > -- > |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| > |: http://libvirt.org -o- http://virt-manager.org :| > |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|