Re: XSA-180 follow-up: repurpose xenconsoled for logging

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 01/06/16 15:00, Wei Liu wrote:
> Hi all
>
> During the discussion of XSA-180 Ian came up with the idea that we
> repurpose xenconsoled to handle logging. I've done some research (and
> some coding as well!).
>
> In my reply to George the other day:
>
>> I just read the code of virtlogd and xenconsoled.
>>
>> I think xenconsoled is missing at least things.
>>
>> From a higher level:
>>
>> 1. Abstraction of rotating file.
>> 2. Abstraction of client.
>> 3. IPC interface to libxl -- presumably we need to create a socket.
>>
> I've done #1 and port existing code to use that -- would be useful in
> general.
>
> #2 is not too hard because xenconsoled already has concept of a domain.
> I suspect some refactoring will be fine.
>
> #3 requires a bit more thinking. Virtlogd has an RPC interface -- I'm
> pretty sure we *don't* want that for xenconsoled. So I spent some time
> this morning and write up a draft for a xenstore based protocol. See
> below.
>
> Also there is an implication here: we put xenconsoled in a really
> critical path. If for some reason it doesn't work all guests are
> blocked. Do we really want to do this?

Sorry to be late to this thread, but I think all my Xen 4.7 tasks are
now done.

Architecturally, this is a bad idea, and -1 from me.

First of all, you are making the assumption that xenconsoled and all
qemus are in the same domain.  This is not necessarily the case,
although I suppose that `xl console` does depend on xl and xenconsoled
being in the same domain.

At the moment, if xenconsoled crashes, the worst that happens is that
you cant interact with guest consoles, and the guests will notice that
their console rings aren't being drained.  This is a safe, albeit
suboptimal, situation which allows guests to continue running unimpeeded.

If qemu is connected via xenconsoled, and xenconsoled crashes, qemu will
block.  Once qemu blocks, the VM is for all intents and purposes, dead. 
As you identify, there is a security implication here, where a guest
which can crash xenconsoled can also DoS all other HVM domains in the
system.

From the scalability side, XenServer supports running 1000 windows VMs
on a single host (subject to sufficient RAM), and xenstored is a big
monlithic single point of failure.  We would like to avoid adding a second.


Managing logging is a hard problem.  However, managing logging is a
system level problem, and needs to fall to the distro package maintainer
to integrate suitably with the distro's expectations, and to use
logrotation/other as applicable.  Introducing custom, project-specific
configuration makes it harder to integrate sensibly, not easier.

I think the best solution here is to manage expectations, not to follow
the kneejerk reaction and hack up a solution.

If someone builds Xen from source and finds later that their disks are
full up with logs, then tough.  If someone installs Xen from a distro,
they should reasonably expect logging, and log management, to be
consistent with other packages from the same distro.

Perhaps providing a default logrotate config file would make things
easier for distro packagers (after all, we provide default
sysvinit/systemd configuration files), and perhaps a written statement
of what gets logged where under which circumstances would make it easier
for people who don't use logrotate, but I think that we shouldn't go any
further than that.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel