Re: [PATCH/RFC 7/9] Virtual network guest device driver

ron minnich
2007-05-11T21:51:13Z

On 5/11/07, Anthony Liguori <aliguori-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org> wrote:

> For low speed devices, I think paravirtualization doesn't make a lot of
> sense unless it's absolutely required. I don't know enough about s390
> to know if it supports things like uarts but if so, then emulating a
> uart would in my mind make a lot more sense than a PV console device.

I don't see how. Paravirtualization is pretty trivial for a console. I
think emulating hardware
is always worth avoiding. A PV ocnsole driver is going to much more
flexible than a uart
emulator.

> This smells a bit like XenStore which I think most will agree was an
> unmitigated disaster.

No, not at all. Just because we represent resources as directories and
files, does that imply or require xenstore? Is /proc a xenstore
entity? Is /sys? Not at all. These resources, which are represented
over 9p as files and directories, are simply a representation of
kernel data structures. I think you are jumping ahead too far, because
that's not what I'm talking about.

What I'm trying to propose is that the kvm host use a standard model
for paravirt resources, and, since we've had 20 years of very good
luck on Plan 9 using 9p and a directory/file model for all resources,
including devices, I am hoping we can use that for the way that kvm
communicates with its guests about devices.

Consider /proc. It works. It's not a thing on disk, or a python glob
like xenstore. There are not even really tree-like data structures in
/proc. The proc outputs are generated on demand as programs do
operations on files in the /proc file system.

This idea is similar -- not the same code, or implementation
technique, but similar.

Our proposal (it was Eric's idea, really, and he has in fact shown it
in practice on IBM hypervisors) is that we define a standard memory
channel for comms, as in Eric's paper; we define a standard
request/response protocol to run over that channel, i.e. 9p, again, as
in Eric's paper(s); and then, what you layer over it is up to the
provider of the resource. This gives us one interface, and it can be
efficient.

Again, in this way, we get a common interface to diverse resources.
This is a basic technique in computer science, and I was sorry to see
Xen ignore it. Eric and I tried to get the Xen team to look at this,
but they were too far along with their myriad interfaces, and it was
too late to change. It's not too late for KVM. I am hoping we can use
this model on KVM, before we have a whole pile of totally different
interfaces to different PV devices.

> This sort of thing gets terribly complicated to
> deal with in the corner cases. Atomic operation of multiple read/write
> operations is difficult to express. Moreover, quite a lot of things are
> naturally expressed as a state machine which is not straight forward to
> do in this sort of model. This may have been all figured out in 9P but
> it's certainly not a simple thing to get right.

We have the QED. It's called Plan 9. Then we have the second QED. It's
called Inferno. They are each a reliable, simple, industrial-strength
kernel running in a router near you. I accept it is hard to get right.
I think you'd have to accept that it can, and in fact has, been gotten
right for quite some time -- 20 years in the case of Plan 9.

> I think a general rule of thumb for a virtualized environment is that
> the closer you stick to the way hardware tends to do things,

You mean like level interrupts emulation in Xen? That was easy? Or not
screwed up? It was one of the messiest things I had to deal with in
the Plan 9 port to Xen. And it made no sense, whatsoever, to have a
level interrupt emulation. Except, of course, that the edge interrupts
were even less fun :-)

I believe that PV can buy us a very clean interface if done right.
Emulating hardware is easy for the simple bits, and very hard to get
perfect for the messy bits .Do we really want to emulate a 10G PHY,
for example?

> Implementing a full 9P client just
> to get console access in something like mini-os would be unfortunate.

9p clients are trivial. newsham's 9p python client is a whopping 352
lines, 20 of them comments. A 9p client is far less code than the sum
of the Linux uart code.

> At least the posted s390 console driver behaves roughly like a uart so
> it's pretty obvious that it will be easy to implement in any OS that
> supports uarts already.

Including all the fifo bugs? Because to really emulate hardware, to
match a driver, you have to correctly emulate the *bugs*, not just the
spec. That's where the fun begins.

I think KVM has a great opportunity here to do a better job than Xen
did with devices. So, I'll keep arguing and see if I can convince you
:-)

thanks

ron