[Qemu-devel] qemu-ga virtio-serial socket clarification

[Qemu-devel] qemu-ga virtio-serial socket clarification

[Matt Broadstone]

Hi,

I've been attempting an experimental qemu agent using a node.js daemon on
the host side, and have run into an issue I was hoping someone here might
be able to help with:

Using libvirt I've set up a 'unix' channel for a domain using virtio-serial
(the same way you would for the existing qemu agent) with the name
'test.agent', in order to bypass libvirt taking ownership of the domain
socket. This works as expected, and so does the following test:

 - [host] $ echo "testing" | nc -U
/var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent
 - [guest] $ cat -v < /dev/virtio-ports/test.agent

Then I tried the same test, converting the host->guest communication to
node.js:

'use strict';
const net = require('net');
const socketPath =
'/var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent';
let socket = net.createConnection(socketPath);
socket.write('testing');

In this case the data makes it across to the guest, however until I
explicitly close the socket on the sender side (`socket.write('testing', ()
=> socket.end())`) both sides block indefinitely. I understand closing the
socket brings the node example to parity with the netcat one, however after
perusing the qemu-ga and libvirt repositories it looks like glib's io
channels are being used on a single socket, and effectively handling
bidirectional data.

Is this the expected behavior?

This would seem to imply that normal async communication over the domain
socket is somehow different in the virtio-serial case (as in I can't
maintain a duplex socket, but would rather have to juggle opening and
closing read/write sockets). In my research I came across another similar
project: https://github.com/xolox/python-negotiator, which requires two
channels: one for host->guest communication, and another for guest->host
communication, likely because of this very issue.

Hopefully someone on this list is more familiar with how this all works and
can point out what I'm missing!

Regards,
Matt Broadstone

Re: [Qemu-devel] qemu-ga virtio-serial socket clarification

[Stefan Hajnoczi]

On Tue, Oct 25, 2016 at 7:14 PM, Matt Broadstone <mbroadst@gmail.com> wrote:
> I've been attempting an experimental qemu agent using a node.js daemon on
> the host side, and have run into an issue I was hoping someone here might
> be able to help with:
>
> Using libvirt I've set up a 'unix' channel for a domain using virtio-serial
> (the same way you would for the existing qemu agent) with the name
> 'test.agent', in order to bypass libvirt taking ownership of the domain
> socket. This works as expected, and so does the following test:
>
>  - [host] $ echo "testing" | nc -U
> /var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent
>  - [guest] $ cat -v < /dev/virtio-ports/test.agent
>
> Then I tried the same test, converting the host->guest communication to
> node.js:
>
> 'use strict';
> const net = require('net');
> const socketPath =
> '/var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent';
> let socket = net.createConnection(socketPath);
> socket.write('testing');
>
> In this case the data makes it across to the guest, however until I
> explicitly close the socket on the sender side (`socket.write('testing', ()
> => socket.end())`) both sides block indefinitely. I understand closing the
> socket brings the node example to parity with the netcat one, however after
> perusing the qemu-ga and libvirt repositories it looks like glib's io
> channels are being used on a single socket, and effectively handling
> bidirectional data.
>
> Is this the expected behavior?
>
> This would seem to imply that normal async communication over the domain
> socket is somehow different in the virtio-serial case (as in I can't
> maintain a duplex socket, but would rather have to juggle opening and
> closing read/write sockets). In my research I came across another similar
> project: https://github.com/xolox/python-negotiator, which requires two
> channels: one for host->guest communication, and another for guest->host
> communication, likely because of this very issue.

virtio-serial is full-duplex.

Please post the receive side test program you are using.

Stefan

Re: [Qemu-devel] qemu-ga virtio-serial socket clarification

[Matt Broadstone]

On Tue, Oct 25, 2016 at 6:27 PM, Stefan Hajnoczi <stefanha@gmail.com> wrote:

> On Tue, Oct 25, 2016 at 7:14 PM, Matt Broadstone <mbroadst@gmail.com>
> wrote:
> > I've been attempting an experimental qemu agent using a node.js daemon on
> > the host side, and have run into an issue I was hoping someone here might
> > be able to help with:
> >
> > Using libvirt I've set up a 'unix' channel for a domain using
> virtio-serial
> > (the same way you would for the existing qemu agent) with the name
> > 'test.agent', in order to bypass libvirt taking ownership of the domain
> > socket. This works as expected, and so does the following test:
> >
> >  - [host] $ echo "testing" | nc -U
> > /var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent
> >  - [guest] $ cat -v < /dev/virtio-ports/test.agent
> >
> > Then I tried the same test, converting the host->guest communication to
> > node.js:
> >
> > 'use strict';
> > const net = require('net');
> > const socketPath =
> > '/var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent';
> > let socket = net.createConnection(socketPath);
> > socket.write('testing');
> >
> > In this case the data makes it across to the guest, however until I
> > explicitly close the socket on the sender side (`socket.write('testing',
> ()
> > => socket.end())`) both sides block indefinitely. I understand closing
> the
> > socket brings the node example to parity with the netcat one, however
> after
> > perusing the qemu-ga and libvirt repositories it looks like glib's io
> > channels are being used on a single socket, and effectively handling
> > bidirectional data.
> >
> > Is this the expected behavior?
> >
> > This would seem to imply that normal async communication over the domain
> > socket is somehow different in the virtio-serial case (as in I can't
> > maintain a duplex socket, but would rather have to juggle opening and
> > closing read/write sockets). In my research I came across another similar
> > project: https://github.com/xolox/python-negotiator, which requires two
> > channels: one for host->guest communication, and another for guest->host
> > communication, likely because of this very issue.
>
> virtio-serial is full-duplex.
>
> Please post the receive side test program you are using.
>
> Stefan
>

Stefan,

The receive side in this case is the same as above: `cat -v <
/dev/virtio-ports/test.agent`, the only variable here is the sending side
changing to the posted node script.

Matt

Re: [Qemu-devel] qemu-ga virtio-serial socket clarification

[Matt Broadstone]

On Tue, Oct 25, 2016 at 7:01 PM, Matt Broadstone <mbroadst@gmail.com> wrote:

> On Tue, Oct 25, 2016 at 6:27 PM, Stefan Hajnoczi <stefanha@gmail.com>
> wrote:
>
>> On Tue, Oct 25, 2016 at 7:14 PM, Matt Broadstone <mbroadst@gmail.com>
>> wrote:
>> > I've been attempting an experimental qemu agent using a node.js daemon
>> on
>> > the host side, and have run into an issue I was hoping someone here
>> might
>> > be able to help with:
>> >
>> > Using libvirt I've set up a 'unix' channel for a domain using
>> virtio-serial
>> > (the same way you would for the existing qemu agent) with the name
>> > 'test.agent', in order to bypass libvirt taking ownership of the domain
>> > socket. This works as expected, and so does the following test:
>> >
>> >  - [host] $ echo "testing" | nc -U
>> > /var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent
>> >  - [guest] $ cat -v < /dev/virtio-ports/test.agent
>> >
>> > Then I tried the same test, converting the host->guest communication to
>> > node.js:
>> >
>> > 'use strict';
>> > const net = require('net');
>> > const socketPath =
>> > '/var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent';
>> > let socket = net.createConnection(socketPath);
>> > socket.write('testing');
>> >
>> > In this case the data makes it across to the guest, however until I
>> > explicitly close the socket on the sender side
>> (`socket.write('testing', ()
>> > => socket.end())`) both sides block indefinitely. I understand closing
>> the
>> > socket brings the node example to parity with the netcat one, however
>> after
>> > perusing the qemu-ga and libvirt repositories it looks like glib's io
>> > channels are being used on a single socket, and effectively handling
>> > bidirectional data.
>> >
>> > Is this the expected behavior?
>> >
>> > This would seem to imply that normal async communication over the domain
>> > socket is somehow different in the virtio-serial case (as in I can't
>> > maintain a duplex socket, but would rather have to juggle opening and
>> > closing read/write sockets). In my research I came across another
>> similar
>> > project: https://github.com/xolox/python-negotiator, which requires two
>> > channels: one for host->guest communication, and another for guest->host
>> > communication, likely because of this very issue.
>>
>> virtio-serial is full-duplex.
>>
>> Please post the receive side test program you are using.
>>
>> Stefan
>>
>
> Stefan,
>
> The receive side in this case is the same as above: `cat -v <
> /dev/virtio-ports/test.agent`, the only variable here is the sending side
> changing to the posted node script.
>
> Matt
>
>
>
Stefan, Michael,

I think what I'm seeing here is that the domain socket interface to
virtio-serial isn't actuall full duplex though. strace is showing me that
the writing side (node.js script => unix domain socket) is actually writing
the data, but then blocking on an epoll read of the socket, I wonder if
this problem is specific to using epoll? I've been digging through the
node/libuv code and it looks like there's no way presently for me to open
the domain socket without polling for reads, so I might be at the end of
the road for this experiment.

Maybe you guys have some other ideas?

Matt

Re: [Qemu-devel] qemu-ga virtio-serial socket clarification

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2709 bytes --]

On Tue, Oct 25, 2016 at 02:14:24PM -0400, Matt Broadstone wrote:
> Hi,
> 
> I've been attempting an experimental qemu agent using a node.js daemon on
> the host side, and have run into an issue I was hoping someone here might
> be able to help with:
> 
> Using libvirt I've set up a 'unix' channel for a domain using virtio-serial
> (the same way you would for the existing qemu agent) with the name
> 'test.agent', in order to bypass libvirt taking ownership of the domain
> socket. This works as expected, and so does the following test:
> 
>  - [host] $ echo "testing" | nc -U
> /var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent
>  - [guest] $ cat -v < /dev/virtio-ports/test.agent
> 
> Then I tried the same test, converting the host->guest communication to
> node.js:
> 
> 'use strict';
> const net = require('net');
> const socketPath =
> '/var/lib/libvirt/qemu/channel/target/domain-T40001/test.agent';
> let socket = net.createConnection(socketPath);
> socket.write('testing');
> 
> In this case the data makes it across to the guest, however until I
> explicitly close the socket on the sender side (`socket.write('testing', ()
> => socket.end())`) both sides block indefinitely. I understand closing the
> socket brings the node example to parity with the netcat one, however after
> perusing the qemu-ga and libvirt repositories it looks like glib's io
> channels are being used on a single socket, and effectively handling
> bidirectional data.
> 
> Is this the expected behavior?

I have reproduced your test and it is expected behavior.

The virtio-serial port inside the guest has two states: connected and
disconnected.  When the port is disconnected read(2) returns 0 (EOF).
When the port is connect read(2) blocks or returns whatever data is
currently available.

Regarding node.js: node.js is an event loop so the process continues
running until your Javascript code terminates the event loop (your
script never does).  Since the virtio-serial port is kept open by the
node.js process on the host, the guest is in the connected state and
read(2) blocks inside the guest.

Here is a ping-pong test with node.js:

'use strict';
const net = require('net');
const socketPath = '/tmp/test.agent';
let socket = net.createConnection(socketPath);
socket.on('data', (data) => {
    let i = Number(data);
    socket.write(i + 1 + '\n');
});
socket.write('0\n');

Inside the guest I can alternate between "cat </dev/vport0p1" and "echo
1" and the node.js script is doing bi-directional communication.  You
could replace those two commands with a similar ping-pong program so it
doesn't require manual interaction.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]