[PATCH] Log qemu-dm's message with date, time and pid

[PATCH] Log qemu-dm's message with date, time and pid

[Yosuke Iwamatsu]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

The qemu-dm's log message doesn't have date, time, and pid. It is
difficult to find out the message we want.

This patch solves the problem without changing the codes which write
messages.

We developed a small program called ioemu-logger. qemu-dm creates a pipe
and starts ioemu-logger. qemu-dm associates a stream with the pipe, and
sets the stream to existing "logfile" variable. When qemu-dm writes a
message to "logfile", ioemu-logger receive it through the pipe and
outputs it with date, time and pid to the actual log file.

Signed-off-by: Yuji Shimada <shimada-yxb@necst.nec.co.jp>

-----------------------
Yosuke Iwamatsu
        NEC Corporation




[-- Attachment #2: ioemu-logger.patch --]
[-- Type: all/allfiles, Size: 7843 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Kevin Wolf]

Yosuke Iwamatsu schrieb:
> We developed a small program called ioemu-logger. qemu-dm creates a pipe
> and starts ioemu-logger. qemu-dm associates a stream with the pipe, and
> sets the stream to existing "logfile" variable. When qemu-dm writes a
> message to "logfile", ioemu-logger receive it through the pipe and
> outputs it with date, time and pid to the actual log file.

Why do you need yet another executable for this rather small piece of
functionality? Shouldn't it be possible to do that all in qemu-dm itself
(and possibly with even less code)?

Kevin

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Ian Jackson]

Yosuke Iwamatsu writes ("[Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> The qemu-dm's log message doesn't have date, time, and pid. It is
> difficult to find out the message we want.

I agree with Kevin Wolf's comments.  But I would go further: why can
this not be done by the code in xend which starts qemu-dm ?  That
would avoid any change to the qemu code.

Also, I think I would discourage the introduction of new features in
tools/ioemu.  I would much prefer submissions to be made against my
new merged qemu tree, which you can currently find at
 http://www.chiark.greenend.org.uk/~ijackson/qemu-xen.git

Ian.

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Daniel P. Berrange]

On Thu, Jun 26, 2008 at 03:46:27PM +0100, Ian Jackson wrote:
> Yosuke Iwamatsu writes ("[Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> > The qemu-dm's log message doesn't have date, time, and pid. It is
> > difficult to find out the message we want.
> 
> I agree with Kevin Wolf's comments.  But I would go further: why can
> this not be done by the code in xend which starts qemu-dm ?  That
> would avoid any change to the qemu code.
> 
> Also, I think I would discourage the introduction of new features in
> tools/ioemu.  I would much prefer submissions to be made against my
> new merged qemu tree, which you can currently find at
>  http://www.chiark.greenend.org.uk/~ijackson/qemu-xen.git

What are your plans wrt to QEMU longer term ?  Are you going to merge this
GIT repo into tools/ioemu in xen-unstable.hg, or thinking its better to
keep a separate tree on an ongoing basis ? Would we have synced up 
releases of the two, or try to apply some ABI stability to libxc and 
libxenstore APIs to allow separate release of QEMU vs XenD tools ? The
latter could be interesting because apps like Xenner are able to use
QEMU-DM in isolation from the rest of the XenD stack.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Yuji Shimada]

Ian Jackson wrote:
> I agree with Kevin Wolf's comments.  But I would go further: why can
> this not be done by the code in xend which starts qemu-dm ?  That
> would avoid any change to the qemu code.

I think that my implementation is more safety way because of the
following reason.

  qemu-dm can easily restart ioemu-logger when ioemu-logger die.
  And ioemu-logger death doesn't affected to guest software.

In addition:

  We developed ioemu-logger without changing the codes which write
  messages. This enables less changes of the codes in qemu-dm.

--
Yuji Shimada

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Kevin Wolf]

Yuji Shimada schrieb:
> Ian Jackson wrote:
>> I agree with Kevin Wolf's comments.  But I would go further: why can
>> this not be done by the code in xend which starts qemu-dm ?  That
>> would avoid any change to the qemu code.

Right, that would be even better.

Or maybe as another option, introduce proper logging functions instead
of fprintf on stderr into (upstream) qemu. This would be the most work,
of course, but IMHO also the cleanest solution.

> I think that my implementation is more safety way because of the
> following reason.
> 
>   qemu-dm can easily restart ioemu-logger when ioemu-logger die.
>   And ioemu-logger death doesn't affected to guest software.

ioemu-logger is so simple that it would be a shame to see it crash anyway.

> In addition:
> 
>   We developed ioemu-logger without changing the codes which write
>   messages. This enables less changes of the codes in qemu-dm.

You could do the same from within qemu-dm or in xend.

Kevin

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Ian Jackson]

Kevin Wolf writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> Or maybe as another option, introduce proper logging functions instead
> of fprintf on stderr into (upstream) qemu. This would be the most work,
> of course, but IMHO also the cleanest solution.

Quite so.

> > I think that my implementation is more safety way because of the
> > following reason.
> > 
> >   qemu-dm can easily restart ioemu-logger when ioemu-logger die.
> >   And ioemu-logger death doesn't affected to guest software.
> 
> ioemu-logger is so simple that it would be a shame to see it crash anyway.

I don't think there is any significant risk of xend dying because it
can't write one of these logs.  xend does lots of logging already of
course.

Ian.

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Yuji Shimada]

I agree with developing proper logging functions. But without these
functions, I think ioemu-logger is a good solution.

We can reboot xend by the following command.
  # /etc/init.d/xend restart
Running of guest domains must not be affected by rebooting xend.

I'm sure that ioemu-logger dying is rare case. But there is no guarantee
that it doesn't die.
If xend starts ioemu-logger, xend loses one end of a pipe by rebooting
xend. After that, xend can't associates stdin for ioemu-logger with the
pipe when xend restarts ioemu-logger after ioemu-logger dies. This
problem happens because the life cycle of xend and qemu-dm is different.
So I think to start ioemu-logger by qemu-dm is more simple.

Using named pipe (FIFO), it's possible to start ioemu-logger by xend.
But sorry, I don't know about python well. Could anyone develope it?

Thanks.

--
Yuji Shimada

On Fri, 27 Jun 2008 14:40:58 +0100
Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:

> Kevin Wolf writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> > Or maybe as another option, introduce proper logging functions instead
> > of fprintf on stderr into (upstream) qemu. This would be the most work,
> > of course, but IMHO also the cleanest solution.
> 
> Quite so.
> 
> > > I think that my implementation is more safety way because of the
> > > following reason.
> > > 
> > >   qemu-dm can easily restart ioemu-logger when ioemu-logger die.
> > >   And ioemu-logger death doesn't affected to guest software.
> > 
> > ioemu-logger is so simple that it would be a shame to see it crash anyway.
> 
> I don't think there is any significant risk of xend dying because it
> can't write one of these logs.  xend does lots of logging already of
> course.
> 
> Ian.

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Kevin Wolf]

Yuji Shimada schrieb:
> I agree with developing proper logging functions. But without these
> functions, I think ioemu-logger is a good solution.
> 
> We can reboot xend by the following command.
>   # /etc/init.d/xend restart
> Running of guest domains must not be affected by rebooting xend.

This seems to be a valid point.

> I'm sure that ioemu-logger dying is rare case. But there is no guarantee
> that it doesn't die.
> If xend starts ioemu-logger, xend loses one end of a pipe by rebooting
> xend. After that, xend can't associates stdin for ioemu-logger with the
> pipe when xend restarts ioemu-logger after ioemu-logger dies. This
> problem happens because the life cycle of xend and qemu-dm is different.
> So I think to start ioemu-logger by qemu-dm is more simple.
> 
> Using named pipe (FIFO), it's possible to start ioemu-logger by xend.
> But sorry, I don't know about python well. Could anyone develope it?

I'm under the impression that you're completely missing my (and Ian's)
point. Actually, I'm not talking about having ioemu-logger started by
xend. I'm talking about not having ioemu-logger at all (at least not as
a new external tool). Xen has already too many of these small tools,
IMHO, and we shouldn't add even more to them without need.

So, please implement it in qemu-dm instead of an external tool, if xend
is not possible.

Kevin

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Yuji Shimada]

OK. I understand what you tell me. Thanks.

I'll consider about a new implementation. It is multithread or
multiprocess programming for logs in qemu-dm without using an external
tool (ioemu-logger).

Thanks.

--
Yuji Shimada

On Mon, 30 Jun 2008 11:45:45 +0200
Kevin Wolf <kwolf@suse.de> wrote:

> Yuji Shimada schrieb:
> > I agree with developing proper logging functions. But without these
> > functions, I think ioemu-logger is a good solution.
> > 
> > We can reboot xend by the following command.
> >   # /etc/init.d/xend restart
> > Running of guest domains must not be affected by rebooting xend.
> 
> This seems to be a valid point.
> 
> > I'm sure that ioemu-logger dying is rare case. But there is no guarantee
> > that it doesn't die.
> > If xend starts ioemu-logger, xend loses one end of a pipe by rebooting
> > xend. After that, xend can't associates stdin for ioemu-logger with the
> > pipe when xend restarts ioemu-logger after ioemu-logger dies. This
> > problem happens because the life cycle of xend and qemu-dm is different.
> > So I think to start ioemu-logger by qemu-dm is more simple.
> > 
> > Using named pipe (FIFO), it's possible to start ioemu-logger by xend.
> > But sorry, I don't know about python well. Could anyone develope it?
> 
> I'm under the impression that you're completely missing my (and Ian's)
> point. Actually, I'm not talking about having ioemu-logger started by
> xend. I'm talking about not having ioemu-logger at all (at least not as
> a new external tool). Xen has already too many of these small tools,
> IMHO, and we shouldn't add even more to them without need.
> 
> So, please implement it in qemu-dm instead of an external tool, if xend
> is not possible.
> 
> Kevin

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Ian Jackson]

Daniel P. Berrange writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> What are your plans wrt to QEMU longer term ?  Are you going to merge this
> GIT repo into tools/ioemu in xen-unstable.hg, or thinking its better to
> keep a separate tree on an ongoing basis ?

No, I don't think it's practical to keep this tree in
xen-unstable.hg.  Everyone else doing work on qemu is using git (or
svn).

>  Would we have synced up 
> releases of the two, or try to apply some ABI stability to libxc and 
> libxenstore APIs to allow separate release of QEMU vs XenD tools ?

Releases of ioemu will follow releases of Xen, but commits will be
made separately.  So there will be at least some compatibility
arrangements.  I'm generally a fan of putting backward compatibility
in interfaces, but I think for now we won't be making any promises.

>  The latter could be interesting because apps like Xenner are able
> to use QEMU-DM in isolation from the rest of the XenD stack.

I certainly don't see any reason to go out of my way to prevent that :-).

Ian.

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Daniel P. Berrange]

On Tue, Jul 01, 2008 at 06:38:25PM +0100, Ian Jackson wrote:
> Daniel P. Berrange writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> > What are your plans wrt to QEMU longer term ?  Are you going to merge this
> > GIT repo into tools/ioemu in xen-unstable.hg, or thinking its better to
> > keep a separate tree on an ongoing basis ?
> 
> No, I don't think it's practical to keep this tree in
> xen-unstable.hg.  Everyone else doing work on qemu is using git (or
> svn).

That's the answer I was hoping for - I much prefer each component
to have its own source tree if the dependancies are clear-cut and
managable separately. Indeed in my ideal universe, every main component
under the tools/ directory in xen-unstable.hg would be independant!

> >  Would we have synced up 
> > releases of the two, or try to apply some ABI stability to libxc and 
> > libxenstore APIs to allow separate release of QEMU vs XenD tools ?
> 
> Releases of ioemu will follow releases of Xen, but commits will be
> made separately.  So there will be at least some compatibility
> arrangements.  I'm generally a fan of putting backward compatibility
> in interfaces, but I think for now we won't be making any promises.

Great, that sounds like a good way to move forward, improving the 
maintainability of the QEMU / ioemu codebase.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [PATCH] Log qemu-dm's message with date, time and pid

[John Levon]

On Tue, Jul 01, 2008 at 06:38:25PM +0100, Ian Jackson wrote:

> Daniel P. Berrange writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> > What are your plans wrt to QEMU longer term ?  Are you going to merge this
> > GIT repo into tools/ioemu in xen-unstable.hg, or thinking its better to
> > keep a separate tree on an ongoing basis ?
> 
> No, I don't think it's practical to keep this tree in
> xen-unstable.hg.  Everyone else doing work on qemu is using git (or
> svn).

Could we host a (read-only) hg mirror on xenbits.xensource.com, though?
All of our tools are built around .hg, and we're in no place to handle
an extra git tree.

regards
john

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Ian Jackson]

John Levon writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> Could we host a (read-only) hg mirror on xenbits.xensource.com, though?
> All of our tools are built around .hg, and we're in no place to handle
> an extra git tree.

Yes, absolutely.

(I haven't looked at the tools available for doing this, to check
whether it's actually possible.  If you have any references to hand
to pass them on; failing that I'll get around to it shortly ...)

Ian.

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Daniel P. Berrange]

On Wed, Jul 02, 2008 at 11:26:47AM +0100, Ian Jackson wrote:
> John Levon writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> > Could we host a (read-only) hg mirror on xenbits.xensource.com, though?
> > All of our tools are built around .hg, and we're in no place to handle
> > an extra git tree.
> 
> Yes, absolutely.
> 
> (I haven't looked at the tools available for doing this, to check
> whether it's actually possible.  If you have any references to hand
> to pass them on; failing that I'll get around to it shortly ...)

A couple of options I know of offhand..

The 'convert' extension to sync from Git to Mercurial

  http://www.selenic.com/mercurial/wiki/index.cgi/ConvertExtension

There is also the Tailor program

  http://progetti.arstecnica.it/tailor

I've not had experiance with using either for the Git -> Mercurial direction
though, so not sure which will work best for Xen's needs.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [PATCH] Log qemu-dm's message with date, time and pid

[Brendan Cully]

On Wednesday, 02 July 2008 at 11:41, Daniel P. Berrange wrote:
> On Wed, Jul 02, 2008 at 11:26:47AM +0100, Ian Jackson wrote:
> > John Levon writes ("Re: [Xen-devel] [PATCH] Log qemu-dm's message with date, time and pid"):
> > > Could we host a (read-only) hg mirror on xenbits.xensource.com, though?
> > > All of our tools are built around .hg, and we're in no place to handle
> > > an extra git tree.
> > 
> > Yes, absolutely.
> > 
> > (I haven't looked at the tools available for doing this, to check
> > whether it's actually possible.  If you have any references to hand
> > to pass them on; failing that I'll get around to it shortly ...)
> 
> A couple of options I know of offhand..
> 
> The 'convert' extension to sync from Git to Mercurial
> 
>   http://www.selenic.com/mercurial/wiki/index.cgi/ConvertExtension
> 
> There is also the Tailor program
> 
>   http://progetti.arstecnica.it/tailor
> 
> I've not had experiance with using either for the Git -> Mercurial direction
> though, so not sure which will work best for Xen's needs.

I'd strongly recommend the convert extension. Tailor is harder to
configure, slower, and last I checked did horrible things to the
changelog DAG. The convert extension is what maintains
http://kernel.org/hg/linux-2.6.