[Qemu-devel] A new direction for vmchannel?

[Qemu-devel] A new direction for vmchannel?

[Anthony Liguori]

The userspace configuration aspects of the current implementation of 
vmchannel are pretty annoying.  Moreover, we would like to make use of 
something like vmchannel in a kernel driver and I fear that it's going 
to be difficult to do that.

So here's an alternative proposal.

Around 2.6.27ish, Eric and I added 9p over virtio support to v9fs.  This 
is all upstream.  We backported the v9fs modules all the way back to 
2.6.18.  I have a 9p client and server library and patches available for 
QEMU.  We were using this for a file system pass through but we could 
also use it as a synthetic file system in the guest (like sysfs).

The guest would just have to mount a directory in a well known location, 
and then you could get vmchannel like semantics by just opening a file 
read/write.  Better yet though would be if we actually exposed vmchannel 
as 9p so that management applications could implement sysfs-like 
hierarchies.

I think there could be a great deal of utility in something like.  For 
portability to Windows (if an app cared), it would have to access the 
mount point through a library of some sort.  We would need a Windows 
virtio-9p driver that exposed the 9p session down to userspace.  We 
could then use our 9p client library in the portability library for Windows.

Virtually all of the code is available for this today, the kernel bits 
are already upstream, there's a reasonable story for Windows, and 
there's very little that the guest can do to get in the way of things.

The only thing that could potentially be an issue is SELinux.  I assume 
you'd have to do an SELinux policy for the guest application anyway 
though so it shouldn't be a problem.

Thoughts?

Regards,

Anthony Liguori

[Qemu-devel] Re: A new direction for vmchannel?

[Chris Wright]

* Anthony Liguori (aliguori@us.ibm.com) wrote:
> The userspace configuration aspects of the current implementation of  
> vmchannel are pretty annoying.  Moreover, we would like to make use of  
> something like vmchannel in a kernel driver and I fear that it's going  
> to be difficult to do that.

What's the use for vmchannel from kernel driver?

> So here's an alternative proposal.
>
> Around 2.6.27ish, Eric and I added 9p over virtio support to v9fs.  This  
> is all upstream.  We backported the v9fs modules all the way back to  
> 2.6.18.  I have a 9p client and server library and patches available for  
> QEMU.  We were using this for a file system pass through but we could  
> also use it as a synthetic file system in the guest (like sysfs).
>
> The guest would just have to mount a directory in a well known location,  
> and then you could get vmchannel like semantics by just opening a file  
> read/write.  Better yet though would be if we actually exposed vmchannel  
> as 9p so that management applications could implement sysfs-like  
> hierarchies.
>
> I think there could be a great deal of utility in something like.  For  
> portability to Windows (if an app cared), it would have to access the  
> mount point through a library of some sort.  We would need a Windows  
> virtio-9p driver that exposed the 9p session down to userspace.  We  
> could then use our 9p client library in the portability library for 
> Windows.
>
> Virtually all of the code is available for this today, the kernel bits  
> are already upstream, there's a reasonable story for Windows, and  
> there's very little that the guest can do to get in the way of things.
>
> The only thing that could potentially be an issue is SELinux.  I assume  
> you'd have to do an SELinux policy for the guest application anyway  
> though so it shouldn't be a problem.
>
> Thoughts?

Heh, works for me ;-)  Last time I suggested an fs it got shot down due to
the burden it puts on the guest implementation (notably windows and
other guests and ease of adding a new fs implementation).

Doesn't directly solve addressing (IOW, easy to do with hierarchical
namespace, but if vmchannel ever talks guest-to-guest...).  Clearly not
a huge issue.

Should handle the reliable messaging bit (one big push for using tcp),
and has advantage of being a structured protocol.

Has the similar ABI issue that we see in Linux with sysfs, namely it's
easy to screw up...but that is manageable.

BTW, what ever happened to just using a serial device (granted needs
some protocol layered on top...)?

thanks,
-chris

[Qemu-devel] Re: A new direction for vmchannel?

[Anthony Liguori]

Chris Wright wrote:
> * Anthony Liguori (aliguori@us.ibm.com) wrote:
>   
>> The userspace configuration aspects of the current implementation of  
>> vmchannel are pretty annoying.  Moreover, we would like to make use of  
>> something like vmchannel in a kernel driver and I fear that it's going  
>> to be difficult to do that.
>>     
>
> What's the use for vmchannel from kernel driver?
>   

Well, it doesn't exactly have to be vmchannel, but the desire is to have 
the management tools to be able to communicate directly with a kernel 
driver.  We could certainly just use virtio and route it through QEMU to 
the management tool.  It would be useful to just use vmchannel though.

>> Thoughts?
>>     
>
> Heh, works for me ;-)  Last time I suggested an fs it got shot down due to
> the burden it puts on the guest implementation (notably windows and
> other guests and ease of adding a new fs implementation).
>
> Doesn't directly solve addressing (IOW, easy to do with hierarchical
> namespace, but if vmchannel ever talks guest-to-guest...).  Clearly not
> a huge issue.
>
> Should handle the reliable messaging bit (one big push for using tcp),
> and has advantage of being a structured protocol.
>
> Has the similar ABI issue that we see in Linux with sysfs, namely it's
> easy to screw up...but that is manageable.
>
> BTW, what ever happened to just using a serial device (granted needs
> some protocol layered on top...)?
>   

I think you end up having a lot of the same issues with respect to 
making it transparent to the guest.  You could still do 9p over serial 
and still use a synthetic file system in the guest.  In fact, this is a 
pretty reasonable thing to do for older kernel guests and even Windows.

Regards,

Anthony Liguori

> thanks,
> -chris
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

[Qemu-devel] Re: A new direction for vmchannel?

[Gleb Natapov]

On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
> Thoughts?
>
Looks good, but I am not to much familiar with p9. Will it provide us
with stream semantics? How much work is needed to support this in
Windows (what is your estimation)? Will migration be transparent to
in guest users?

--
			Gleb.

[Qemu-devel] Re: A new direction for vmchannel?

[Anthony Liguori]

Gleb Natapov wrote:
> On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
>   
>> Thoughts?
>>
>>     
> Looks good, but I am not to much familiar with p9. Will it provide us
> with stream semantics?

Sure.  It has read and write operations.  You just have to implement 
them in the same sort of way you'd implement them for a character device.

>  How much work is needed to support this in
> Windows (what is your estimation)?

If you structure your guest applications to use a library, sort of like 
libsysfs, then on Windows, you could implement a 9P client in 
userspace.  I have a 9P client that can be used for this.  You just need 
some way to get the stream to userspace.  You could write a virtio 
windows driver that exposed the stream down to userspace.  You could 
also use an alternative transport for Windows (like a serial port).

>  Will migration be transparent to
> in guest users?
>   

There's no better migration story for vmchannel backends implemented 
outside of QEMU.  For the ones in QEMU, migration should be transparent.

Regards,

Anthony Liguori

> --
> 			Gleb.
>   

Re: [Qemu-devel] Re: A new direction for vmchannel?

[Dor Laor]

Anthony Liguori wrote:
> Gleb Natapov wrote:
>> On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
>>  
>>> Thoughts?
>>>
>>>     
>> Looks good, but I am not to much familiar with p9. Will it provide us
>> with stream semantics?
>
> Sure.  It has read and write operations.  You just have to implement 
> them in the same sort of way you'd implement them for a character device.

So why not having a virtio pv-serial device.
Above this device/driver users can have several implementations/semantics:
    - raw messages
    - 9p
    - userspace tcp/ip
       - We already used the host part of it.
       - It provides reliability and handles migration too.
       - Since its userspace, there is no routing/addressing involved.
The above will make windows and other OSs happy and users can still get 
9p or tcp or raw.
Maybe even the virtio console can be based over it (Don't really know 
nothing about it )

btw: I do suggest to keep the former work of nic based vmchannel. Except 
for the cmdline,
every thing else is committed. It might be good for older OSs and just 
work out of the box.


>
>>  How much work is needed to support this in
>> Windows (what is your estimation)?
>
> If you structure your guest applications to use a library, sort of 
> like libsysfs, then on Windows, you could implement a 9P client in 
> userspace.  I have a 9P client that can be used for this.  You just 
> need some way to get the stream to userspace.  You could write a 
> virtio windows driver that exposed the stream down to userspace.  You 
> could also use an alternative transport for Windows (like a serial port).
>
>>  Will migration be transparent to
>> in guest users?
>>   
>
> There's no better migration story for vmchannel backends implemented 
> outside of QEMU.  For the ones in QEMU, migration should be transparent.
>
> Regards,
>
> Anthony Liguori
>
>> -- 
>>             Gleb.
>>   
>
>
>

Re: [Qemu-devel] Re: A new direction for vmchannel?

[Alexander Graf]

On 24.01.2009, at 01:02, Dor Laor <dlaor@redhat.com> wrote:

> Anthony Liguori wrote:
>> Gleb Natapov wrote:
>>> On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
>>>
>>>> Thoughts?
>>>>
>>>>
>>> Looks good, but I am not to much familiar with p9. Will it provide  
>>> us
>>> with stream semantics?
>>
>> Sure.  It has read and write operations.  You just have to  
>> implement them in the same sort of way you'd implement them for a  
>> character device.
>
> So why not having a virtio pv-serial device.
> Above this device/driver users can have several implementations/ 
> semantics:
>   - raw messages
>   - 9p
>   - userspace tcp/ip
>      - We already used the host part of it.
>      - It provides reliability and handles migration too.
>      - Since its userspace, there is no routing/addressing involved.
> The above will make windows and other OSs happy and users can still  
> get 9p or tcp or raw.
> Maybe even the virtio console can be based over it (Don't really  
> know nothing about it )

I don't see where there's any benefit here. You need a driver in the  
guest for virtio either way - and what exactly goes to userspace ( fs  
or datagram protocol ) is an implementation detail, no?

Alex

>
>
> btw: I do suggest to keep the former work of nic based vmchannel.  
> Except for the cmdline,
> every thing else is committed. It might be good for older OSs and  
> just work out of the box.
>
>
>>
>>> How much work is needed to support this in
>>> Windows (what is your estimation)?
>>
>> If you structure your guest applications to use a library, sort of  
>> like libsysfs, then on Windows, you could implement a 9P client in  
>> userspace.  I have a 9P client that can be used for this.  You just  
>> need some way to get the stream to userspace.  You could write a  
>> virtio windows driver that exposed the stream down to userspace.   
>> You could also use an alternative transport for Windows (like a  
>> serial port).
>>
>>> Will migration be transparent to
>>> in guest users?
>>>
>>
>> There's no better migration story for vmchannel backends  
>> implemented outside of QEMU.  For the ones in QEMU, migration  
>> should be transparent.
>>
>> Regards,
>>
>> Anthony Liguori
>>
>>> -- 
>>>            Gleb.
>>>
>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] Re: A new direction for vmchannel?

[Dor Laor]

Alexander Graf wrote:
>
>
>
>
> On 24.01.2009, at 01:02, Dor Laor <dlaor@redhat.com> wrote:
>
>> Anthony Liguori wrote:
>>> Gleb Natapov wrote:
>>>> On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
>>>>
>>>>> Thoughts?
>>>>>
>>>>>
>>>> Looks good, but I am not to much familiar with p9. Will it provide us
>>>> with stream semantics?
>>>
>>> Sure.  It has read and write operations.  You just have to implement 
>>> them in the same sort of way you'd implement them for a character 
>>> device.
>>
>> So why not having a virtio pv-serial device.
>> Above this device/driver users can have several 
>> implementations/semantics:
>>   - raw messages
>>   - 9p
>>   - userspace tcp/ip
>>      - We already used the host part of it.
>>      - It provides reliability and handles migration too.
>>      - Since its userspace, there is no routing/addressing involved.
>> The above will make windows and other OSs happy and users can still 
>> get 9p or tcp or raw.
>> Maybe even the virtio console can be based over it (Don't really know 
>> nothing about it )
>
> I don't see where there's any benefit here. You need a driver in the 
> guest for virtio either way - and what exactly goes to userspace ( fs 
> or datagram protocol ) is an implementation detail, no?
>
There are two benefits:
a. Simplify the kernel driver over virtio.
    It does not need to implement 9p interface (important for non Linux) 
or other.
    It would be also possible to use the serial as a medium, so no 
driver at all.
b. Flexibility - you're not bound to a specific solution.
    One can use userspace tcp/ip over it, others will use 9p (which is 
nice), etc.

IMO if the current 9p is mature and supports stream based poll semantics 
like was discussed, and is not too
complex for Windows to implement we can use 9p.
Otherwise, let's go for a generic solution like pv serial using virtio.

- dor

> Alex
>
>>
>>
>> btw: I do suggest to keep the former work of nic based vmchannel. 
>> Except for the cmdline,
>> every thing else is committed. It might be good for older OSs and 
>> just work out of the box.
>>
>>
>>>
>>>> How much work is needed to support this in
>>>> Windows (what is your estimation)?
>>>
>>> If you structure your guest applications to use a library, sort of 
>>> like libsysfs, then on Windows, you could implement a 9P client in 
>>> userspace.  I have a 9P client that can be used for this.  You just 
>>> need some way to get the stream to userspace.  You could write a 
>>> virtio windows driver that exposed the stream down to userspace.  
>>> You could also use an alternative transport for Windows (like a 
>>> serial port).
>>>
>>>> Will migration be transparent to
>>>> in guest users?
>>>>
>>>
>>> There's no better migration story for vmchannel backends implemented 
>>> outside of QEMU.  For the ones in QEMU, migration should be 
>>> transparent.
>>>
>>> Regards,
>>>
>>> Anthony Liguori
>>>
>>>> -- 
>>>>            Gleb.
>>>>
>>>
>>>
>>>
>>
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> -- 
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Qemu-devel] Re: A new direction for vmchannel?

[Daniel P. Berrange]

On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
> The userspace configuration aspects of the current implementation of 
> vmchannel are pretty annoying.  Moreover, we would like to make use of 
> something like vmchannel in a kernel driver and I fear that it's going 
> to be difficult to do that.
> 
> So here's an alternative proposal.
> 
> Around 2.6.27ish, Eric and I added 9p over virtio support to v9fs.  This 
> is all upstream.  We backported the v9fs modules all the way back to 
> 2.6.18.  I have a 9p client and server library and patches available for 
> QEMU.  We were using this for a file system pass through but we could 
> also use it as a synthetic file system in the guest (like sysfs).
> 
> The guest would just have to mount a directory in a well known location, 
> and then you could get vmchannel like semantics by just opening a file 
> read/write.  Better yet though would be if we actually exposed vmchannel 
> as 9p so that management applications could implement sysfs-like 
> hierarchies.
> 
> I think there could be a great deal of utility in something like.  For 
> portability to Windows (if an app cared), it would have to access the 
> mount point through a library of some sort.  We would need a Windows 
> virtio-9p driver that exposed the 9p session down to userspace.  We 
> could then use our 9p client library in the portability library for Windows.
> 
> Virtually all of the code is available for this today, the kernel bits 
> are already upstream, there's a reasonable story for Windows, and 
> there's very little that the guest can do to get in the way of things.
> 
> The only thing that could potentially be an issue is SELinux.  I assume 
> you'd have to do an SELinux policy for the guest application anyway 
> though so it shouldn't be a problem.

For use cases where you are exposing metadata from the host to the guest
this would be a very convenient approach indeed. As asked elsewhere in this
thread, my main thought would be about how well it suits a application that
wants a generic stream based connection between host & guest ? Efficient 
integration into a poll(2) based event loop would be key to that. Regular
files don't offer that kind of ability ordinarily, and not clear whether 
fifo's would be provided for in p9fs between host/guest ?

In any case, if we have a usable p9fs backend for QEMU, I don't see why we 
shouldn't integrate that in QEMU, regardless of whether it serves the more 
general vmchannel use cases. Sharing filesystems is an interesting idea in
its own right after all. 

I also really don't like guest deployment / configuration complexity that
is accompanying the NIC device based vmchannel approach. There are just 
far too many things that can go wrong with it wrt the guest OS & apps using
networking. IMHO, the core motivation of vmchannel is to have a secure 
guest <->  host data transport that can we relied upon from the moment 
guest userspace appears, preferrably with zero guest admin configuration
requirements, and no need for authentication keys to establish guest 
identity. UNIX domain sockets are a great example of this ideal, providing 
a reliable data stream for the localhost before network makes any appearance,
with builtin client authentication via SCM_CREDS.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

[Qemu-devel] Re: A new direction for vmchannel?

[Anthony Liguori]

Daniel P. Berrange wrote:
> On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
>   
>> The userspace configuration aspects of the current implementation of 
>> vmchannel are pretty annoying.  Moreover, we would like to make use of 
>> something like vmchannel in a kernel driver and I fear that it's going 
>> to be difficult to do that.
>>
>> So here's an alternative proposal.
>>
>> Around 2.6.27ish, Eric and I added 9p over virtio support to v9fs.  This 
>> is all upstream.  We backported the v9fs modules all the way back to 
>> 2.6.18.  I have a 9p client and server library and patches available for 
>> QEMU.  We were using this for a file system pass through but we could 
>> also use it as a synthetic file system in the guest (like sysfs).
>>
>> The guest would just have to mount a directory in a well known location, 
>> and then you could get vmchannel like semantics by just opening a file 
>> read/write.  Better yet though would be if we actually exposed vmchannel 
>> as 9p so that management applications could implement sysfs-like 
>> hierarchies.
>>
>> I think there could be a great deal of utility in something like.  For 
>> portability to Windows (if an app cared), it would have to access the 
>> mount point through a library of some sort.  We would need a Windows 
>> virtio-9p driver that exposed the 9p session down to userspace.  We 
>> could then use our 9p client library in the portability library for Windows.
>>
>> Virtually all of the code is available for this today, the kernel bits 
>> are already upstream, there's a reasonable story for Windows, and 
>> there's very little that the guest can do to get in the way of things.
>>
>> The only thing that could potentially be an issue is SELinux.  I assume 
>> you'd have to do an SELinux policy for the guest application anyway 
>> though so it shouldn't be a problem.
>>     
>
> For use cases where you are exposing metadata from the host to the guest
> this would be a very convenient approach indeed. As asked elsewhere in this
> thread, my main thought would be about how well it suits a application that
> wants a generic stream based connection between host & guest ? Efficient 
> integration into a poll(2) based event loop would be key to that.

You mean for a very large number of files (determining which property 
has changed?).

The way you would do this today, without special inotify support, is to 
have a special file in the hierarchy called "change-notify".  You can 
write a list of deliminated files and whenever one of those files 
becomes readable, the file itself will become readable (returning a 
deliminated list of files that have changed since last read).

This way, you get a file you can select on for a very large number of 
files.  That said, it would be nice to add proper inotify support to 
v9fs too.

>  Regular
> files don't offer that kind of ability ordinarily, and not clear whether 
> fifo's would be provided for in p9fs between host/guest ?
>   

I'm going to put together a patch this weekend and I'll include a 
streaming example.  Basically, you just ignore the file offset and 
read/write to the file to your heart's content.

Regards,

Anthony Liguori

> In any case, if we have a usable p9fs backend for QEMU, I don't see why we 
> shouldn't integrate that in QEMU, regardless of whether it serves the more 
> general vmchannel use cases. Sharing filesystems is an interesting idea in
> its own right after all. 
>
> I also really don't like guest deployment / configuration complexity that
> is accompanying the NIC device based vmchannel approach. There are just 
> far too many things that can go wrong with it wrt the guest OS & apps using
> networking. IMHO, the core motivation of vmchannel is to have a secure 
> guest <->  host data transport that can we relied upon from the moment 
> guest userspace appears, preferrably with zero guest admin configuration
> requirements, and no need for authentication keys to establish guest 
> identity. UNIX domain sockets are a great example of this ideal, providing 
> a reliable data stream for the localhost before network makes any appearance,
> with builtin client authentication via SCM_CREDS.
>
> Regards,
> Daniel
>   

[Qemu-devel] Re: A new direction for vmchannel?

[Gleb Natapov]

On Sat, Jan 24, 2009 at 11:52:06AM -0600, Anthony Liguori wrote:
>> For use cases where you are exposing metadata from the host to the guest
>> this would be a very convenient approach indeed. As asked elsewhere in this
>> thread, my main thought would be about how well it suits a application that
>> wants a generic stream based connection between host & guest ? 
>> Efficient integration into a poll(2) based event loop would be key to 
>> that.
>
> You mean for a very large number of files (determining which property  
> has changed?).
>
I think what Daniel means is that for file to have stream semantic it is
not enough to ignore offset on read/write, but poll also should behave
similar to how it behaves with char device fd. With regular files poll
will always report that fd is ready for I/O.

--
			Gleb.

[Qemu-devel] Re: A new direction for vmchannel?

[Anthony Liguori]

Gleb Natapov wrote:
> On Sat, Jan 24, 2009 at 11:52:06AM -0600, Anthony Liguori wrote:
>   
>>> For use cases where you are exposing metadata from the host to the guest
>>> this would be a very convenient approach indeed. As asked elsewhere in this
>>> thread, my main thought would be about how well it suits a application that
>>> wants a generic stream based connection between host & guest ? 
>>> Efficient integration into a poll(2) based event loop would be key to 
>>> that.
>>>       
>> You mean for a very large number of files (determining which property  
>> has changed?).
>>
>>     
> I think what Daniel means is that for file to have stream semantic it is
> not enough to ignore offset on read/write, but poll also should behave
> similar to how it behaves with char device fd. With regular files poll
> will always report that fd is ready for I/O.
>   

I'm not really sure, but given the 9P protocol, I suspect it would have 
to.  Eric?

Regards,

Anthony Liguori

> --
> 			Gleb.
>   

[Qemu-devel] Re: A new direction for vmchannel?

[Anthony Liguori]

Gleb Natapov wrote:
> On Sat, Jan 24, 2009 at 11:52:06AM -0600, Anthony Liguori wrote:
>   
>>> For use cases where you are exposing metadata from the host to the guest
>>> this would be a very convenient approach indeed. As asked elsewhere in this
>>> thread, my main thought would be about how well it suits a application that
>>> wants a generic stream based connection between host & guest ? 
>>> Efficient integration into a poll(2) based event loop would be key to 
>>> that.
>>>       
>> You mean for a very large number of files (determining which property  
>> has changed?).
>>
>>     
> I think what Daniel means is that for file to have stream semantic it is
> not enough to ignore offset on read/write, but poll also should behave
> similar to how it behaves with char device fd. With regular files poll
> will always report that fd is ready for I/O.
>   

Thinking more about this, the difficulty is that poll() only has useful 
semantics when you're dealing with a buffered stream of some sort.  That 
is, poll() is only really capable of asking whether there is data 
pending in your read buffer.

With 9P, you have to explicitly send a read request.  You can implement 
buffered IO by simply sending constant read requests such that there is 
always one read request pending.  I don't think it's useful to do this 
in the kernel.

Unfortunately, there's no way to do async IO in userspace that doesn't 
suck so that would make this pretty difficult.  We could use a thread 
pool, but that's somewhat soul crushing and doesn't scale well.  I think 
that puts a requirement on v9fs to support linux-aio.

Regards,

Anthony Liguori

> --
> 			Gleb.
>   

Re: [Qemu-devel] Re: A new direction for vmchannel?

[Jamie Lokier]

Anthony Liguori wrote:
> Thinking more about this, the difficulty is that poll() only has useful 
> semantics when you're dealing with a buffered stream of some sort.  That 
> is, poll() is only really capable of asking whether there is data 
> pending in your read buffer.
> 
> With 9P, you have to explicitly send a read request.  You can implement 
> buffered IO by simply sending constant read requests such that there is 
> always one read request pending.  I don't think it's useful to do this 
> in the kernel.

In linux/Documentation/fs/9p.txt:

  nodevmap      do not map special files - represent them as normal files.
                This can be used to share devices/named pipes/sockets between
                hosts.  This functionality will be expanded in later versions.

Does this mean it can provide the semantics of a named pipe
or unix domain socket over 9p?

Both would provide suitable poll() behaviour.  A unix domain socket
sounds especially useful, practically a drop in replacement for the
current network vmchannel, without configuration downsides.

But what about non-Linux guests?

> Unfortunately, there's no way to do async IO in userspace that doesn't 
> suck so that would make this pretty difficult.  We could use a thread 
> pool, but that's somewhat soul crushing and doesn't scale well.  I think 
> that puts a requirement on v9fs to support linux-aio.

It sounds like a weird mashup of unix regular file behaviour and unix
socket behaviour.  Poll doesn't work, aio might, but you can
read/write like a stream?  Very un-unix.

Even if it can be made to work, that's sufficiently un-unix to reduce
its general utility with existing scripting languages, existing
monitoring programs etc.

-- Jamie

Re: [Qemu-devel] Re: A new direction for vmchannel?

[Anthony Liguori]

Jamie Lokier wrote:
> Anthony Liguori wrote:
>   
>> Thinking more about this, the difficulty is that poll() only has useful 
>> semantics when you're dealing with a buffered stream of some sort.  That 
>> is, poll() is only really capable of asking whether there is data 
>> pending in your read buffer.
>>
>> With 9P, you have to explicitly send a read request.  You can implement 
>> buffered IO by simply sending constant read requests such that there is 
>> always one read request pending.  I don't think it's useful to do this 
>> in the kernel.
>>     
>
> In linux/Documentation/fs/9p.txt:
>
>   nodevmap      do not map special files - represent them as normal files.
>                 This can be used to share devices/named pipes/sockets between
>                 hosts.  This functionality will be expanded in later versions.
>
> Does this mean it can provide the semantics of a named pipe
> or unix domain socket over 9p?
>   

Eric will have to correct me if I'm wrong, but I believe that that bit 
of functionality is not yet implemented (although it's been discussed).

> But what about non-Linux guests?
>   

The plan for non-Linux guests is to expose the 9P protocol stream down 
to userspace and provide a library with posix-like file system 
semantics.  For instance kvmfs_open(), kvmfs_read(), kvmfs_write(), 
etc.  Applications that care about portability can use this library on 
Linux and non-Linux.

For non-Linux guests where we can get a kernel driver in, the driver 
will feed the 9P stream directly to userspace.  For guests that we 
cannot get a kernel driver in, we can stream 9P to the guest via TCP, 
serial port, etc.  This requires some config in the guest's userspace 
and may require some admin intervention but that's the price you pay for 
avoiding a kernel module.

>> Unfortunately, there's no way to do async IO in userspace that doesn't 
>> suck so that would make this pretty difficult.  We could use a thread 
>> pool, but that's somewhat soul crushing and doesn't scale well.  I think 
>> that puts a requirement on v9fs to support linux-aio.
>>     
>
> It sounds like a weird mashup of unix regular file behaviour and unix
> socket behaviour.  Poll doesn't work, aio might, but you can
> read/write like a stream?  Very un-unix.
>   

Right.  We can certainly modify v9fs to make poll() behave more naturally.

Regards,

Anthony Liguori

> Even if it can be made to work, that's sufficiently un-unix to reduce
> its general utility with existing scripting languages, existing
> monitoring programs etc.
>
> -- Jamie
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

[Qemu-devel] Re: A new direction for vmchannel?

[Daniel P. Berrange]

On Sat, Jan 24, 2009 at 11:52:06AM -0600, Anthony Liguori wrote:
> Daniel P. Berrange wrote:
> >On Fri, Jan 23, 2009 at 08:45:33AM -0600, Anthony Liguori wrote:
> >
> >For use cases where you are exposing metadata from the host to the guest
> >this would be a very convenient approach indeed. As asked elsewhere in this
> >thread, my main thought would be about how well it suits a application that
> >wants a generic stream based connection between host & guest ? Efficient 
> >integration into a poll(2) based event loop would be key to that.
> 
> You mean for a very large number of files (determining which property 
> has changed?).

Two aspects to main loop integration - for the filesystem as structured
metadata use case, an efficient notifiction mechanism for discovering 
changes to files is key. For the stream based connection use case, 
obviously polling for incoming data / output write availabilty.
 
> The way you would do this today, without special inotify support, is to 
> have a special file in the hierarchy called "change-notify".  You can 
> write a list of deliminated files and whenever one of those files 
> becomes readable, the file itself will become readable (returning a 
> deliminated list of files that have changed since last read).
>
> This way, you get a file you can select on for a very large number of 
> files.  That said, it would be nice to add proper inotify support to 
> v9fs too.

There's probably two distinct user requirements here. For apps which need
to implement something that'd work with vmchannel for both Linux and
Windows guests, the special 'change-notify' file might be more convenient.
For apps which want to work with native Linux APIs, then real inotify 
support would be desirable, since that's a standard interface across 
all filesystems in Linux.

> > Regular
> >files don't offer that kind of ability ordinarily, and not clear whether 
> >fifo's would be provided for in p9fs between host/guest ?
> >  
> 
> I'm going to put together a patch this weekend and I'll include a 
> streaming example.  Basically, you just ignore the file offset and 
> read/write to the file to your heart's content.

Basically my use case would be that you have an existing application
that supports UNIX domain sockets, and / or TCP sockets, and you wish
to also have it run acros the host <-> guest channel. Although we're
not using sockets here, being able to easily integrate into an app 
written around a sockets model is the high level goal.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

[Qemu-devel] Re: A new direction for vmchannel?

[Anthony Liguori]

Daniel P. Berrange wrote:
> On Sat, Jan 24, 2009 at 11:52:06AM -0600, Anthony Liguori wrote:
>   
>>> Regular
>>> files don't offer that kind of ability ordinarily, and not clear whether 
>>> fifo's would be provided for in p9fs between host/guest ?
>>>  
>>>       
>> I'm going to put together a patch this weekend and I'll include a 
>> streaming example.  Basically, you just ignore the file offset and 
>> read/write to the file to your heart's content.
>>     
>
> Basically my use case would be that you have an existing application
> that supports UNIX domain sockets, and / or TCP sockets, and you wish
> to also have it run acros the host <-> guest channel. Although we're
> not using sockets here, being able to easily integrate into an app 
> written around a sockets model is the high level goal.
>   

It would be pretty trivial to have a small guest daemon that runs and 
makes a file system stream available as a unix socket.  For instance, 
you would have:

/qemufs/org/libvirt/qemud/stream0

The daemon would create a unix socket, 
/qemufs/org/libvirt/qemud/stream0.sock, and would spawn a thread that 
constantly read from /qemufs/org/libvirt/qemud/stream0 providing the 
data in the stream0.sock's buffer.  I think that's a pretty reasonable 
compromise if that interface is strongly required.

Maybe even just a library to provide that level of functionality?  So 
you could have a function that was essentially 
vmc_open_as_stream("/org/libvirt/qemud/stream0") and the library would 
spawn a thread and create a unix domain socketpair() and return the 
appropriate end.  That eliminates the need to have a daemon running.

Regards,

Anthony Liguori

Regards,

Anthony Liguori

> Daniel
>