From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: Possible bug with huge unflushed console buffer Date: Wed, 08 Aug 2012 17:42:23 +0100 Message-ID: References: <20120808174355.470746e0@internecto.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20120808174355.470746e0@internecto.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Mark van Dijk , xen-devel@lists.xen.org Cc: Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org On 08/08/2012 16:43, "Mark van Dijk" wrote: > Greetings list, > > Yesterday the following scenario took place. In short, I had a HVM host > with a poorly configured firewall. It logged way more than it should, > and it had been doing that for days. IPTables entries with -j LOG > show up in the console. I told Konrad about this but I said it was a PV > host while in fact it is concerning a HVM host. > > Anyway, when I saw that this was the case I corrected iptables and then > I removed those faulty log entries from the logfiles in /var/log. > Finally I issued /etc/init.d/rsyslogd reload. > > Suddenly the server stopped responding at all, not even arp requests > were answered. I opened up a console via xl (none was open before > this) and I was greeted with a crazy amount of log entries - it took > about five or six minutes before the whole buffer was flushed to my > screen. Then the server responded again. > > Today I viewed syslog's messages in detail and found this and a lot of > similar entries: > > http://pastebin.com/Ga7aE7hb > > So the questions that I have now are - Is this a bug? How is console > history handled and where is the console's unread history stored? > Somewhere in the hypervisor or in dom0 or domU memory space? Is there a > limit to how much info can be stored? > > The behaviour I encountered i.e. a system lock seems to suggest that > there was some kind of buffer overflow. And this can be achieved by > something as simple as 'iptables -I INPUT -j LOG'. Perhaps it is a wise > idea to consider xl console -c to clear the console's history, then at > least I could exit the console and clear unsent messages... Is your VM logging to its virtual serial line? You could just not do that... I think this is due to the guest's qemu-dm process stuffing its transmitted serial data into a pty, to be consumed by 'xl console', and if that doesn't happen the buffers fill up, serial processing stops, and we back up all the way to the guest. -- Keir > I am not receiving xen-devel messages, so please CC me on replies. > Thank you.