[Qemu-devel] questions about host side of virtio-serial

[Qemu-devel] questions about host side of virtio-serial

[Chris Friesen]

Hi,

I'm working on a native user of virtio-serial (ie, not going via the 
qemu guest agent).

The information at "http://www.linux-kvm.org/page/Virtio-serial_API" 
does a good job of describing the guest side of things, but has very 
little information about the host side of things.

In particular, assuming that the host side is using a chardev mapped to 
a unix socket:

1) Is there any way for the host app to get information about whether or 
not the guest is reading the messages?  (i.e. logically equivalent to 
getting POLLHUP in the guest when the host app disconnects.)

2) Suppose the host sends a large message.  The guest app reads a 
portion of the message, then crashes.  We respawn the guest app and 
start reading again, but now we're in the middle of a message of 
arbitrary size.  Is there a recommended technique to re-sync the host 
and guest?

3) Same as 2, but the guest sending to the host and the host app 
crashing partway through.

4) If nothing in the guest is reading the data, how much data can the 
host send before it will get an error?  Is there a way to adjust this?

Thanks,
Chris

Re: [Qemu-devel] questions about host side of virtio-serial

[Richard W.M. Jones]

On Wed, Jul 30, 2014 at 12:52:41PM -0600, Chris Friesen wrote:
> Hi,
> 
> I'm working on a native user of virtio-serial (ie, not going via the
> qemu guest agent).
> 
> The information at "http://www.linux-kvm.org/page/Virtio-serial_API"
> does a good job of describing the guest side of things, but has very
> little information about the host side of things.

You might also want to read the libguestfs source code, since
libguestfs is a major and long-time user of virtio-serial.

In particular these files:

  src/launch-direct.c
  src/proto.c
  src/conn-socket.c
  daemon/proto.c
  src/launch-libvirt.c  # if interested in using virtio-serial from libvirt

> In particular, assuming that the host side is using a chardev mapped
> to a unix socket:
> 
> 1) Is there any way for the host app to get information about
> whether or not the guest is reading the messages?  (i.e. logically
> equivalent to getting POLLHUP in the guest when the host app
> disconnects.)

No, I don't believe that is possible.  It acts like a real serial port
and throws away bytes when no one is listening (on both ends).

> 2) Suppose the host sends a large message.  The guest app reads a
> portion of the message, then crashes.  We respawn the guest app and
> start reading again, but now we're in the middle of a message of
> arbitrary size.  Is there a recommended technique to re-sync the
> host and guest?

AFAIK that's either very difficult or impossible.  Maybe with some
kind of self-synchronizing protocol, or if you ran SLIP/PPP on top of
the raw virtio-serial channel?

In the libguestfs case we wouldn't even try to go there -- if
something in the VM crashes we completely recreate the virtual machine
from scratch.

> 3) Same as 2, but the guest sending to the host and the host app
> crashing partway through.
> 
> 4) If nothing in the guest is reading the data, how much data can
> the host send before it will get an error?

It won't get an error - the sender will block.  Except on ARM where
there is a race condition in virtio-mmio causing writes to be thrown
away (https://bugs.launchpad.net/qemu/+bug/1224444).

> Is there a way to adjust this?

Not as far as I know.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Re: [Qemu-devel] questions about host side of virtio-serial

[Chris Friesen]

On 07/31/2014 03:32 AM, Richard W.M. Jones wrote:
> On Wed, Jul 30, 2014 at 12:52:41PM -0600, Chris Friesen wrote:

>> In particular, assuming that the host side is using a chardev mapped
>> to a unix socket:
>>
>> 1) Is there any way for the host app to get information about
>> whether or not the guest is reading the messages?  (i.e. logically
>> equivalent to getting POLLHUP in the guest when the host app
>> disconnects.)
>
> No, I don't believe that is possible.  It acts like a real serial port
> and throws away bytes when no one is listening (on both ends).

With qemu 1.4.2 it blocks or returns EAGAIN if the guest tries to write 
to the char device, but the behaviour is different if the host tries to 
write to the unix socket.  See 4 below.


>> 4) If nothing in the guest is reading the data, how much data can
>> the host send before it will get an error?
>
> It won't get an error - the sender will block.  Except on ARM where
> there is a race condition in virtio-mmio causing writes to be thrown
> away (https://bugs.launchpad.net/qemu/+bug/1224444).

I'm seeing behaviour that doesn't match what you describe.  With nothing 
listening in the guest I can send a packet to the unix socket on the 
host and it gets accepted:

connect(7, {sa_family=AF_FILE, 
sun_path="/var/lib/libvirt/qemu/cgcs.messaging.instance-00000004.sock"}, 
61) = 0
sendto(7, "\r\360\255\272N\0instcgcs.server_grp\0cgcs.s"..., 78, 0, 
NULL, 0) = 78


Then, up in the guest I start a new process to read from the device and 
I can read the data that was sent in by the host.

open("/dev/virtio-ports/cgcs.messaging", O_RDWR|O_NONBLOCK) = 3
read(3, "\r\360\255\272N\0instcgcs.server_grp\0cgcs.s"..., 10000) = 78



This is a VM started by openstack, the pertinent parameters are:

-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -chardev 
socket,id=charchannel0,path=/var/lib/libvirt/qemu/cgcs.messaging.instance-00000004.sock,server,nowait 
-device 
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=cgcs.messaging


Chris

Re: [Qemu-devel] questions about host side of virtio-serial

[Paolo Bonzini]

Il 31/07/2014 16:45, Chris Friesen ha scritto:
> With qemu 1.4.2 it blocks or returns EAGAIN if the guest tries to write
> to the char device, but the behaviour is different if the host tries to
> write to the unix socket.  See 4 below.

1.4.2 is too old.  Even CentOS 6's franken-0.12 would be preferrable to
such an old release, in particular it had the flow control patches from
1.5.0 backported.

Paolo