[Qemu-devel] Breaking out virtfs as a standalone server?

[Qemu-devel] Breaking out virtfs as a standalone server?

[Rob Landley]

Right now, there's no decent userspace server for the 9p filesystem that
I can find.  (In part because the 9P2000.L spec is an undocumented work
in progress.)

The only up-to-date server seems to be virtfs in qemu, which has no TCP
transport layer.

Are there any plans to:

A) Add a TCP transport layer so we can test with something we can
intercept/examine/log/redirect with netcat and such?

B) Break the 9p server out so it could be built as a standalone
userspace program?

Rob

Re: [Qemu-devel] Breaking out virtfs as a standalone server?

[Venkateswararao Jujjuri]

On 04/11/2011 06:42 AM, Rob Landley wrote:
> Right now, there's no decent userspace server for the 9p filesystem that
> I can find.  (In part because the 9P2000.L spec is an undocumented work
> in progress.)
This statement is true for 9P2000.L protocol;
But for older protocols we have standalone servers like spfs/npfs.
http://sourceforge.net/projects/npfs/
http://9p.cat-v.org/implementations

> The only up-to-date server seems to be virtfs in qemu, which has no TCP
> transport layer.
>
> Are there any plans to:
>
> A) Add a TCP transport layer so we can test with something we can
> intercept/examine/log/redirect with netcat and such?
No plans as of now; I know folks in the Latchesar Ionkov attempted char 
dev transport.
Not sure the latest though.

http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTim4eZttAmaNQfOuM1h7cmLvO-osckHNunMvG7o%2B%40mail.gmail.com&forum_name=v9fs-developer
> B) Break the 9p server out so it could be built as a standalone
> userspace program?

No plans yet..and I think this is a bigger discussion.
Being part of QEMU brings few implicit advantages like simplicity in 
sharing, security
and performance advantage. I think taking it out can have its own merits.

If there is enough interest I am sure these two are something we can 
look at as a community.

- JV

> Rob
>

Re: [Qemu-devel] [V9fs-developer] Breaking out virtfs as a standalone server?

[Jim Garlick]

FWIW I've put a prerelease (1.0-pre20) of a pure 9P2000.L server up here:

http://code.google.com/p/diod/

If anyone would like to give it a spin with a 2.6.38-ish kernel,
please do and let me know how it goes! 

To get started with a quick (insecure) test:

$ tar xjf diod-1.0-pre20.tar.bz2
$ cd diod-1.0-pre20
$ ./configure --disable-munge
$ make
$ sudo diod/diod -n -f -l 0.0.0.0:564 -c /dev/null -e /tmp
$ sudo utils/diodmount  -n -oport=564 localhost:/tmp /mnt

I've been testing on a RHEL 6 (glibc-2.12) based system.
We're just starting to test on big clusters with NFS.
We've been successful with home directories and other things, but
there is still much testing to do before a real 1.0 release.

Also, as I mentioned before, I have attempted to document 9P2000.L
as I've implemented it (referring to the kernel and qemu source)
on a wiki page here: 

   http://code.google.com/p/diod/wiki/protocol

Maybe that will be a helpful start for someone implementing a new
server, until something more official comes along.

Jim

On Mon, Apr 11, 2011 at 01:28:49PM -0700, Venkateswararao Jujjuri wrote:
> On 04/11/2011 06:42 AM, Rob Landley wrote:
> > Right now, there's no decent userspace server for the 9p filesystem that
> > I can find.  (In part because the 9P2000.L spec is an undocumented work
> > in progress.)
> This statement is true for 9P2000.L protocol;
> But for older protocols we have standalone servers like spfs/npfs.
> http://sourceforge.net/projects/npfs/
> http://9p.cat-v.org/implementations
> 
> > The only up-to-date server seems to be virtfs in qemu, which has no TCP
> > transport layer.
> >
> > Are there any plans to:
> >
> > A) Add a TCP transport layer so we can test with something we can
> > intercept/examine/log/redirect with netcat and such?
> No plans as of now; I know folks in the Latchesar Ionkov attempted char 
> dev transport.
> Not sure the latest though.
> 
> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTim4eZttAmaNQfOuM1h7cmLvO-osckHNunMvG7o%2B%40mail.gmail.com&forum_name=v9fs-developer
> > B) Break the 9p server out so it could be built as a standalone
> > userspace program?
> 
> No plans yet..and I think this is a bigger discussion.
> Being part of QEMU brings few implicit advantages like simplicity in 
> sharing, security
> and performance advantage. I think taking it out can have its own merits.
> 
> If there is enough interest I am sure these two are something we can 
> look at as a community.
> 
> - JV
> 
> > Rob
> >
> 
> 
> ------------------------------------------------------------------------------
> Forrester Wave Report - Recovery time is now measured in hours and minutes
> not days. Key insights are discussed in the 2010 Forrester Wave Report as
> part of an in-depth evaluation of disaster recovery service providers.
> Forrester found the best-in-class provider in terms of services and vision.
> Read this report now!  http://p.sf.net/sfu/ibm-webcastpromo
> _______________________________________________
> V9fs-developer mailing list
> V9fs-developer@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/v9fs-developer

Re: [Qemu-devel] Breaking out virtfs as a standalone server?

[Rob Landley]

On 04/11/2011 03:28 PM, Venkateswararao Jujjuri wrote:
> On 04/11/2011 06:42 AM, Rob Landley wrote:
>> Right now, there's no decent userspace server for the 9p filesystem that
>> I can find.  (In part because the 9P2000.L spec is an undocumented work
>> in progress.)
> This statement is true for 9P2000.L protocol;

According to my research on the topic, anyway:

  http://landley.livejournal.com/48698.html

> But for older protocols we have standalone servers like spfs/npfs.
> http://sourceforge.net/projects/npfs/

This would be the one that has no documentation or web page, builds
without ./configure, and has no way to specify which directory to export
but apparently can ONLY export / on the host?

> http://9p.cat-v.org/implementations

I looked at those.  Several are unfinished libraries, some also only
export / and treat the whole "restrict what you're exporting to a
subdir" problem as inherently unsolvable (readlink -f), the main one
everybody seems to test with is the Inferno equivalent of usermode
Linux, there's a python server that refuses to run without some third
party encryption/authentication library that it never OCCURRED to their
developers you might want t disable...

I also tracked down more like http://code.google.com/p/diod/ (and
emailed its author for a while: that project is stalled due to his
desire to rewrite large chunks of it, and a lack of time).

I also subscribed to v9fs-users, which is not the world's highest
traffic list:

  http://sourceforge.net/mailarchive/forum.php?forum_name=v9fs-users

>> The only up-to-date server seems to be virtfs in qemu, which has no TCP
>> transport layer.
>>
>> Are there any plans to:
>>
>> A) Add a TCP transport layer so we can test with something we can
>> intercept/examine/log/redirect with netcat and such?
>
> No plans as of now; I know folks in the Latchesar Ionkov attempted char
> dev transport.

NFS works over TCP.  Samba works over TCP.  But not p9.  Is there some
reason to go out of the way to avoid it?

> Not sure the latest though.
> 
> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTim4eZttAmaNQfOuM1h7cmLvO-osckHNunMvG7o%2B%40mail.gmail.com&forum_name=v9fs-developer

Ah, if I want information about userspace servers I should subscribe to
v9fs-developer instead of v9fs-user, because it has no users yet.  Got it.

>> B) Break the 9p server out so it could be built as a standalone
>> userspace program?
> 
> No plans yet..and I think this is a bigger discussion.

If this filesystem is to become more than an academic exercise, it needs
a server that can export a specific directory.

> Being part of QEMU brings few implicit advantages like simplicity in
> sharing, security
> and performance advantage. I think taking it out can have its own merits.

I wasn't suggesting removing it from QEMU.  Having it in QEMU is great,
the code is written and works, it's useful as-is, and half the _idea_ of
this is that it's simpler than samba or NFS (which is damning with faint
praise, I know).

In theory, all the actual protocol encoding and decoding (and doing the
read/write/stat stuff on the host) is a single C file, correct?  Right
now there isn't any reference implementation of that server-side code,
but there is a working example of it.  Virtfs is a working example, just
not hooked up to a particularly useful transport.  (If it had a TCP
transport I could route it back out through a tap interface or -redir
port and use qemu as a test server... but it doesn't.  As far as I can
tell, virtio is intentionally the _least_ flexible mechanism for that
sort of thing.  Although maybe there's docs on this and I've just missed
them...)

Rob

Re: [Qemu-devel] Breaking out virtfs as a standalone server?

[Venkateswararao Jujjuri]

On 04/11/2011 10:52 PM, Rob Landley wrote:
> On 04/11/2011 03:28 PM, Venkateswararao Jujjuri wrote:
>> On 04/11/2011 06:42 AM, Rob Landley wrote:
>>> Right now, there's no decent userspace server for the 9p filesystem that
>>> I can find.  (In part because the 9P2000.L spec is an undocumented work
>>> in progress.)
>> This statement is true for 9P2000.L protocol;
> According to my research on the topic, anyway:
>
>    http://landley.livejournal.com/48698.html

You wrote " And at the moment write support seems to be broken for me. 
But I was able to mount a directory from the host system and cat a file, 
which is progress."

Can you please explain where it is broken. IT should be working good.

- JV

>> But for older protocols we have standalone servers like spfs/npfs.
>> http://sourceforge.net/projects/npfs/
> This would be the one that has no documentation or web page, builds
> without ./configure, and has no way to specify which directory to export
> but apparently can ONLY export / on the host?
>
>> http://9p.cat-v.org/implementations
> I looked at those.  Several are unfinished libraries, some also only
> export / and treat the whole "restrict what you're exporting to a
> subdir" problem as inherently unsolvable (readlink -f), the main one
> everybody seems to test with is the Inferno equivalent of usermode
> Linux, there's a python server that refuses to run without some third
> party encryption/authentication library that it never OCCURRED to their
> developers you might want t disable...
>
> I also tracked down more like http://code.google.com/p/diod/ (and
> emailed its author for a while: that project is stalled due to his
> desire to rewrite large chunks of it, and a lack of time).
>
> I also subscribed to v9fs-users, which is not the world's highest
> traffic list:
>
>    http://sourceforge.net/mailarchive/forum.php?forum_name=v9fs-users
>
>>> The only up-to-date server seems to be virtfs in qemu, which has no TCP
>>> transport layer.
>>>
>>> Are there any plans to:
>>>
>>> A) Add a TCP transport layer so we can test with something we can
>>> intercept/examine/log/redirect with netcat and such?
>> No plans as of now; I know folks in the Latchesar Ionkov attempted char
>> dev transport.
> NFS works over TCP.  Samba works over TCP.  But not p9.  Is there some
> reason to go out of the way to avoid it?
>
>> Not sure the latest though.
>>
>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTim4eZttAmaNQfOuM1h7cmLvO-osckHNunMvG7o%2B%40mail.gmail.com&forum_name=v9fs-developer
> Ah, if I want information about userspace servers I should subscribe to
> v9fs-developer instead of v9fs-user, because it has no users yet.  Got it.
>
>>> B) Break the 9p server out so it could be built as a standalone
>>> userspace program?
>> No plans yet..and I think this is a bigger discussion.
> If this filesystem is to become more than an academic exercise, it needs
> a server that can export a specific directory.
>
>> Being part of QEMU brings few implicit advantages like simplicity in
>> sharing, security
>> and performance advantage. I think taking it out can have its own merits.
> I wasn't suggesting removing it from QEMU.  Having it in QEMU is great,
> the code is written and works, it's useful as-is, and half the _idea_ of
> this is that it's simpler than samba or NFS (which is damning with faint
> praise, I know).
>
> In theory, all the actual protocol encoding and decoding (and doing the
> read/write/stat stuff on the host) is a single C file, correct?  Right
> now there isn't any reference implementation of that server-side code,
> but there is a working example of it.  Virtfs is a working example, just
> not hooked up to a particularly useful transport.  (If it had a TCP
> transport I could route it back out through a tap interface or -redir
> port and use qemu as a test server... but it doesn't.  As far as I can
> tell, virtio is intentionally the _least_ flexible mechanism for that
> sort of thing.  Although maybe there's docs on this and I've just missed
> them...)
>
> Rob
>

Re: [Qemu-devel] [V9fs-developer] Breaking out virtfs as a standalone server?

[Rob Landley]

On 04/12/2011 09:34 AM, Venkateswararao Jujjuri wrote:
> On 04/11/2011 10:52 PM, Rob Landley wrote:
>> On 04/11/2011 03:28 PM, Venkateswararao Jujjuri wrote:
>>> On 04/11/2011 06:42 AM, Rob Landley wrote:
>>>> Right now, there's no decent userspace server for the 9p filesystem that
>>>> I can find.  (In part because the 9P2000.L spec is an undocumented work
>>>> in progress.)
>>> This statement is true for 9P2000.L protocol;
>> According to my research on the topic, anyway:
>>
>>    http://landley.livejournal.com/48698.html
> 
> You wrote " And at the moment write support seems to be broken for me. 
> But I was able to mount a directory from the host system and cat a file, 
> which is progress."
> 
> Can you please explain where it is broken. IT should be working good.

That's an old blog entry, we already dealt with that a while back:

  http://sourceforge.net/mailarchive/message.php?msg_id=26950979

What I hadn't found was a better server, until the new diod release.
(Hence my interest in possibly getting the code in qemu factored out and
more flexible.)

Rob