9pfs developers docs

9pfs developers docs

[Christian Schoenebeck]

Hi,

I started setting up some developer documentation for 9pfs:

	https://wiki.qemu.org/Documentation/9p

Still quite a bunch that should be added (e.g. there should be a section about 
threads and coroutines), but at least it's a start ...

Best regards,
Christian Schoenebeck

Re: 9pfs developers docs

[Greg Kurz]

On Sun, 31 Jan 2021 19:23:52 +0100
Christian Schoenebeck <qemu_oss@crudebyte.com> wrote:

> Hi,
> 

Hi Christian,

> I started setting up some developer documentation for 9pfs:
> 
> 	https://wiki.qemu.org/Documentation/9p
> 
> Still quite a bunch that should be added (e.g. there should be a section about 
> threads and coroutines), but at least it's a start ...
> 

I agree that a bunch of other things should be documented, but that's
definitely a great start. Thanks for doing this !

Just one remark on the topology diagram:

https://wiki.qemu.org/File:9pfs_topology.png

It gives the impression that the 9p transport and server can
handle multiple guests, which they certainly don't : each
9p server lives in exactly one device which is exposed to
exactly one guest.

Cheers,

--
Greg

> Best regards,
> Christian Schoenebeck
> 
> 
> 
> 

Re: 9pfs developers docs

[Christian Schoenebeck]

On Montag, 1. Februar 2021 10:24:26 CET Greg Kurz wrote:
> On Sun, 31 Jan 2021 19:23:52 +0100
> 
> Christian Schoenebeck <qemu_oss@crudebyte.com> wrote:
> > Hi,
> 
> Hi Christian,
> 
> > I started setting up some developer documentation for 9pfs:
> > 	https://wiki.qemu.org/Documentation/9p
> > 
> > Still quite a bunch that should be added (e.g. there should be a section
> > about threads and coroutines), but at least it's a start ...
> 
> I agree that a bunch of other things should be documented, but that's
> definitely a great start. Thanks for doing this !
> 
> Just one remark on the topology diagram:
> 
> https://wiki.qemu.org/File:9pfs_topology.png
> 
> It gives the impression that the 9p transport and server can
> handle multiple guests, which they certainly don't : each
> 9p server lives in exactly one device which is exposed to
> exactly one guest.

Right, I haven't considered that the diagram might be interpreted that way. My 
primary intention was to show the 3 main components of 9pfs from design 
perspective and secondary showing that multiple guests can share storage.

So what would be better: a) duplicating the server side in the diagram (then 
the image might become a bit large in height), b) dropping the multiple 
guests, c) making the issue with server instances clear in the text?

If there are other things that you might think should be outlined by 
additional diagram(s) let me know, then I can add that in one rush.

--

BTW I'm no longer able to run the 'local' 9p tests, --slow doesn't work for 
me. If you don't have an idea what I might be missing, then I have to look why 
the CLI parameter is not interpreted.

Best regards,
Christian Schoenebeck

Re: 9pfs developers docs

[Greg Kurz]

On Mon, 01 Feb 2021 12:30:52 +0100
Christian Schoenebeck <qemu_oss@crudebyte.com> wrote:

> On Montag, 1. Februar 2021 10:24:26 CET Greg Kurz wrote:
> > On Sun, 31 Jan 2021 19:23:52 +0100
> > 
> > Christian Schoenebeck <qemu_oss@crudebyte.com> wrote:
> > > Hi,
> > 
> > Hi Christian,
> > 
> > > I started setting up some developer documentation for 9pfs:
> > > 	https://wiki.qemu.org/Documentation/9p
> > > 
> > > Still quite a bunch that should be added (e.g. there should be a section
> > > about threads and coroutines), but at least it's a start ...
> > 
> > I agree that a bunch of other things should be documented, but that's
> > definitely a great start. Thanks for doing this !
> > 
> > Just one remark on the topology diagram:
> > 
> > https://wiki.qemu.org/File:9pfs_topology.png
> > 
> > It gives the impression that the 9p transport and server can
> > handle multiple guests, which they certainly don't : each
> > 9p server lives in exactly one device which is exposed to
> > exactly one guest.
> 
> Right, I haven't considered that the diagram might be interpreted that way. My 
> primary intention was to show the 3 main components of 9pfs from design 
> perspective and secondary showing that multiple guests can share storage.
> 
> So what would be better: a) duplicating the server side in the diagram (then 
> the image might become a bit large in height), b) dropping the multiple 
> guests, c) making the issue with server instances clear in the text?

I'd rather go for b)

> 
> If there are other things that you might think should be outlined by 
> additional diagram(s) let me know, then I can add that in one rush.
> 
> --
> 
> BTW I'm no longer able to run the 'local' 9p tests, --slow doesn't work for 
> me. If you don't have an idea what I might be missing, then I have to look why 
> the CLI parameter is not interpreted.
> 

Is it that '-m slow' doesn't work when running 'qos-test' or
that 'make check-qtest SPEEP=slow' doesn't run the slow tests ?

The latter was discussed on IRC last year but I don't know if
anyone has tried to investigate this yet.

Nov 24 11:36:53 <groug>	th_huth, Hi. FYI it seems that the meson conversion kinda broke 'make check SPEED=slow'. Test programs aren't passed '-m slow'
Nov 24 11:51:42 <f4bug>	th_huth: do you know who uses/tests SPEED=slow?
Nov 24 11:52:03 <f4bug>	th_huth: I thought this was a block-related feature
Nov 24 11:52:44 <groug>	f4bug, it is supposedly used by gitlab CI
Nov 24 11:52:59 <groug>	.gitlab-ci.yml:    MAKE_CHECK_ARGS: check-qtest SPEED=slow
Nov 24 12:50:53 <th_huth>	groug, I'm also running make check SPEED=slow manually sometimes ... I guess that got lost in the conversion to ninja ... bonzini, did you ever try?
Nov 24 12:51:03 <bonzini>	no it shouldn't
Nov 24 12:51:21 <th_huth>	let me check...
Nov 24 12:51:40 <bonzini>	ah, the tests are chosen correctly but -m slow is lost
Nov 24 12:52:02 <groug>	yes that's what I see
Nov 24 12:54:04 <groug>	bonzini, missing bits in scripts/mtest2make.py ?
Nov 24 12:54:28 <bonzini>	groug: sort of, but assuming that all executables support -m slow wouldn't work

Cc'ing Thomas and Paolo for additional details.

> Best regards,
> Christian Schoenebeck
> 
> 

Re: 9pfs developers docs

[Paolo Bonzini]

On 01/02/21 13:26, Greg Kurz wrote:
> Is it that '-m slow' doesn't work when running 'qos-test' or
> that 'make check-qtest SPEEP=slow' doesn't run the slow tests ?
> 
> The latter was discussed on IRC last year but I don't know if
> anyone has tried to investigate this yet.
> 
> Nov 24 11:36:53 <groug>	th_huth, Hi. FYI it seems that the meson conversion kinda broke 'make check SPEED=slow'. Test programs aren't passed '-m slow'
> Nov 24 11:51:42 <f4bug>	th_huth: do you know who uses/tests SPEED=slow?
> Nov 24 11:52:03 <f4bug>	th_huth: I thought this was a block-related feature
> Nov 24 11:52:44 <groug>	f4bug, it is supposedly used by gitlab CI
> Nov 24 11:52:59 <groug>	.gitlab-ci.yml:    MAKE_CHECK_ARGS: check-qtest SPEED=slow
> Nov 24 12:50:53 <th_huth>	groug, I'm also running make check SPEED=slow manually sometimes ... I guess that got lost in the conversion to ninja ... bonzini, did you ever try?
> Nov 24 12:51:03 <bonzini>	no it shouldn't
> Nov 24 12:51:21 <th_huth>	let me check...
> Nov 24 12:51:40 <bonzini>	ah, the tests are chosen correctly but -m slow is lost
> Nov 24 12:52:02 <groug>	yes that's what I see
> Nov 24 12:54:04 <groug>	bonzini, missing bits in scripts/mtest2make.py ?
> Nov 24 12:54:28 <bonzini>	groug: sort of, but assuming that all executables support -m slow wouldn't work
> 
> Cc'ing Thomas and Paolo for additional details.

Basically what's in the IRC log is still true.  I'll look into using an 
environment variable "SPEED" in addition to g_test_{slow,quick} in order 
to restore SPEED=slow behavior.

Paolo

Re: 9pfs developers docs

[qemu_oss--- via]

On Montag, 1. Februar 2021 13:26:49 CET Greg Kurz wrote:
> On Mon, 01 Feb 2021 12:30:52 +0100
> 
> Christian Schoenebeck <qemu_oss@crudebyte.com> wrote:
> > On Montag, 1. Februar 2021 10:24:26 CET Greg Kurz wrote:
> > > On Sun, 31 Jan 2021 19:23:52 +0100
> > > 
> > > Christian Schoenebeck <qemu_oss@crudebyte.com> wrote:
> > > > Hi,
> > > 
> > > Hi Christian,
> > > 
> > > > I started setting up some developer documentation for 9pfs:
> > > > 	https://wiki.qemu.org/Documentation/9p
> > > > 
> > > > Still quite a bunch that should be added (e.g. there should be a
> > > > section
> > > > about threads and coroutines), but at least it's a start ...
> > > 
> > > I agree that a bunch of other things should be documented, but that's
> > > definitely a great start. Thanks for doing this !
> > > 
> > > Just one remark on the topology diagram:
> > > 
> > > https://wiki.qemu.org/File:9pfs_topology.png
> > > 
> > > It gives the impression that the 9p transport and server can
> > > handle multiple guests, which they certainly don't : each
> > > 9p server lives in exactly one device which is exposed to
> > > exactly one guest.
> > 
> > Right, I haven't considered that the diagram might be interpreted that
> > way. My primary intention was to show the 3 main components of 9pfs from
> > design perspective and secondary showing that multiple guests can share
> > storage.
> > 
> > So what would be better: a) duplicating the server side in the diagram
> > (then the image might become a bit large in height), b) dropping the
> > multiple guests, c) making the issue with server instances clear in the
> > text?
> I'd rather go for b)

Updated the diagram on the wiki page.

To keep noise low, I won't send emails on further changes to that wiki page. 
If you want to be auto notified then just add yourself to the watch list 
there.

> > If there are other things that you might think should be outlined by
> > additional diagram(s) let me know, then I can add that in one rush.
> > 
> > --
> > 
> > BTW I'm no longer able to run the 'local' 9p tests, --slow doesn't work
> > for
> > me. If you don't have an idea what I might be missing, then I have to look
> > why the CLI parameter is not interpreted.
> 
> Is it that '-m slow' doesn't work when running 'qos-test' or
> that 'make check-qtest SPEEP=slow' doesn't run the slow tests ?

Ah, that's '-m slow', not '--slow'. Yeah, that works for qos-test. I added the 
'-m slow' switch to the wiki page as well.

For now I can live with that, as I am more commonly calling qos-test directly. 
But it would be nice if the slow tests would make it into the general chain of 
all QEMU tests accordingly again.

> The latter was discussed on IRC last year but I don't know if
> anyone has tried to investigate this yet.
> 
> Nov 24 11:36:53 <groug>	th_huth, Hi. FYI it seems that the meson conversion
> kinda broke 'make check SPEED=slow'. Test programs aren't passed '-m slow'
> Nov 24 11:51:42 <f4bug>	th_huth: do you know who uses/tests SPEED=slow? Nov
> 24 11:52:03 <f4bug>	th_huth: I thought this was a block-related feature 
Nov
> 24 11:52:44 <groug>	f4bug, it is supposedly used by gitlab CI
> Nov 24 11:52:59 <groug>	.gitlab-ci.yml:    MAKE_CHECK_ARGS: check-qtest
> SPEED=slow Nov 24 12:50:53 <th_huth>	groug, I'm also running make check
> SPEED=slow manually sometimes ... I guess that got lost in the conversion
> to ninja ... bonzini, did you ever try? Nov 24 12:51:03 <bonzini>	no it
> shouldn't
> Nov 24 12:51:21 <th_huth>	let me check...
> Nov 24 12:51:40 <bonzini>	ah, the tests are chosen correctly but -m slow 
is
> lost Nov 24 12:52:02 <groug>	yes that's what I see
> Nov 24 12:54:04 <groug>	bonzini, missing bits in scripts/mtest2make.py ?
> Nov 24 12:54:28 <bonzini>	groug: sort of, but assuming that all 
executables
> support -m slow wouldn't work
> 
> Cc'ing Thomas and Paolo for additional details.
> 
> > Best regards,
> > Christian Schoenebeck

Re: 9pfs developers docs

[Christian Schoenebeck]

On Montag, 1. Februar 2021 14:37:02 CET qemu_oss--- via wrote:
> On Montag, 1. Februar 2021 13:26:49 CET Greg Kurz wrote:
> > > > I agree that a bunch of other things should be documented, but that's
> > > > definitely a great start. Thanks for doing this !
> > > > 
> > > > Just one remark on the topology diagram:
> > > > 
> > > > https://wiki.qemu.org/File:9pfs_topology.png
> > > > 
> > > > It gives the impression that the 9p transport and server can
> > > > handle multiple guests, which they certainly don't : each
> > > > 9p server lives in exactly one device which is exposed to
> > > > exactly one guest.
> > > 
> > > Right, I haven't considered that the diagram might be interpreted that
> > > way. My primary intention was to show the 3 main components of 9pfs from
> > > design perspective and secondary showing that multiple guests can share
> > > storage.
> > > 
> > > So what would be better: a) duplicating the server side in the diagram
> > > (then the image might become a bit large in height), b) dropping the
> > > multiple guests, c) making the issue with server instances clear in the
> > > text?
> > 
> > I'd rather go for b)
> 
> Updated the diagram on the wiki page.
> 
> To keep noise low, I won't send emails on further changes to that wiki page.
> If you want to be auto notified then just add yourself to the watch list
> there.

Ok, I think I'm done with this page:
https://wiki.qemu.org/Documentation/9p

At least I think I've described everything I wanted to from my side regarding 
current 9p status.

Best regards,
Christian Schoenebeck

Re: 9pfs developers docs

[Greg Kurz]

On Thu, 25 Feb 2021 17:31:22 +0100
Christian Schoenebeck <qemu_oss@crudebyte.com> wrote:

> On Montag, 1. Februar 2021 14:37:02 CET qemu_oss--- via wrote:
> > On Montag, 1. Februar 2021 13:26:49 CET Greg Kurz wrote:
> > > > > I agree that a bunch of other things should be documented, but that's
> > > > > definitely a great start. Thanks for doing this !
> > > > > 
> > > > > Just one remark on the topology diagram:
> > > > > 
> > > > > https://wiki.qemu.org/File:9pfs_topology.png
> > > > > 
> > > > > It gives the impression that the 9p transport and server can
> > > > > handle multiple guests, which they certainly don't : each
> > > > > 9p server lives in exactly one device which is exposed to
> > > > > exactly one guest.
> > > > 
> > > > Right, I haven't considered that the diagram might be interpreted that
> > > > way. My primary intention was to show the 3 main components of 9pfs from
> > > > design perspective and secondary showing that multiple guests can share
> > > > storage.
> > > > 
> > > > So what would be better: a) duplicating the server side in the diagram
> > > > (then the image might become a bit large in height), b) dropping the
> > > > multiple guests, c) making the issue with server instances clear in the
> > > > text?
> > > 
> > > I'd rather go for b)
> > 
> > Updated the diagram on the wiki page.
> > 
> > To keep noise low, I won't send emails on further changes to that wiki page.
> > If you want to be auto notified then just add yourself to the watch list
> > there.
> 
> Ok, I think I'm done with this page:
> https://wiki.qemu.org/Documentation/9p
> 
> At least I think I've described everything I wanted to from my side regarding 
> current 9p status.
> 

Great thanks for doing this ! I'll have a look ASAP.

Cheers,

--
Greg


> Best regards,
> Christian Schoenebeck
> 
> 

Re: 9pfs developers docs

[Peter Maydell]

On Sun, 31 Jan 2021 at 18:25, Christian Schoenebeck
<qemu_oss@crudebyte.com> wrote:
> I started setting up some developer documentation for 9pfs:
>
>         https://wiki.qemu.org/Documentation/9p
>
> Still quite a bunch that should be added (e.g. there should be a section about
> threads and coroutines), but at least it's a start ...

Thanks for working on this; our developer docs of QEMU internals
are very patchy and improvements and additions to them are very welcome.

The wiki's a good place for initial work on the docs, since it
allows quick iteration and collaboration. When you think the docs
are in a more final state you might consider moving them into
docs/devel/ as an rst file, so they become part of the developer
manual.

thanks
-- PMM

Re: 9pfs developers docs

[Christian Schoenebeck]

On Montag, 1. Februar 2021 12:42:44 CET Peter Maydell wrote:
> On Sun, 31 Jan 2021 at 18:25, Christian Schoenebeck
> 
> <qemu_oss@crudebyte.com> wrote:
> > I started setting up some developer documentation for 9pfs:
> >         https://wiki.qemu.org/Documentation/9p
> > 
> > Still quite a bunch that should be added (e.g. there should be a section
> > about threads and coroutines), but at least it's a start ...
> 
> Thanks for working on this; our developer docs of QEMU internals
> are very patchy and improvements and additions to them are very welcome.
> 
> The wiki's a good place for initial work on the docs, since it
> allows quick iteration and collaboration. When you think the docs
> are in a more final state you might consider moving them into
> docs/devel/ as an rst file, so they become part of the developer
> manual.
> 
> thanks
> -- PMM

I wonder how that is solved on kernel.org, because they also have a wiki and 
in-tree rst files. Maybe they have some way to merge content between the two 
in a sensible way.

Best regards,
Christian Schoenebeck