Re: libxl: error handling before xenstored runs

[Ian Campbell <Ian.Campbell@eu.citrix.com>]

On Thu, 2011-02-10 at 21:55 +0000, Vincent Hanquez wrote: 
> On 10/02/11 11:24, Ian Campbell wrote:
> > Right but this approach doesn't work with xenstored in a stubdomain.
> 
> yeah I know. xenstored in a stubdom is just an experiment, when it 
> become a serious feature, this argument would hold. however it's not 
> going to be use in 4.1, and in any production settings.

Accepted.

As I understand it people are actively using stub-xenstored in
disaggregation research today. Reinvigorating the stub domain approach
for Xen is also (going to be) one of our GSoC proposals this year.

> > Part of the point of using the ring protocol even when this isn't the
> > case is to help ensure that it is possible and help avoid regressions
> > etc.
> >
> >> The protocol is not design to do async either, so leaving unconsumed
> >> request, could be pretty disastrous if the other end show up. Providing
> >> the kernel doesn't detect it (i don't think it does [1]), it would imply
> >> spurious reply, for example the previous waiting read on "/abc/def"
> >> could reply to a next read on "/xyz/123".
> >
> > The wire protocol includes a req_id which is echoed in the response
> > which sh/could facilitate multiplexing this sort of thing. The pvops
> > kernel currently always sets it to zero but that's just an
> > implementation detail ;-) Currently the kernel does (roughly):
> 
> The kernel is not the one exclusively setting the rid. this is a client 
> initialized value. any xs implementation can use it any way they want 
> (including the kernel implementation).
> 
> Turns out that most of the implementations are actually putting rid to 0 
> anyway (the ocaml and C implementation are, the windows one isn't).
> 
> Even then, if you could initialize it to some value, what value is that 
> going to be ? there's just no way to know if someone else is not using 
> this rid already globally (since the ring is a global OS thing). Which 
> basically would means tracking pid (the kernel meaning) along with the rid ?

Since the kernel mediates all access to the actual ring it can handle
the req_id with a single incrementing integer and fake out whatever is
necessary to its users.

It's trivial to fixup the in-kernel xs users. Most likely it only
involves changing a single function in the core xs kernel code which
everyone else must use anyway.

For userspace users it's a little trickier but the kernel just needs to
remember the userspace supplied req_id before inserting its own and to
reverse the substitution in the reply. If you were to support multiple
outstanding active requests then it would be natural to stash the id in
whatever data structure you were using for that. If you only want to
simplify by only supporting a single active request (by throwing away
responses to aborted/timed out requests as I suggested earlier) you only
need to remember the user provided req_id for the one request which is
trivial.

AFAIK we don't have any kernel code which does clever things such as
using a pointer to a datastructure as the req_id but if we did (and we
were unwilling to simply change it) then the userspace solution would
work there too.

> >>> Maybe we should add an explicit ping/pong ring message to the xs ring
> >>> protocol?
> >>
> >> And who's going to reply to this if xenstored is missing ? you would
> >> require the kernel to introspect the messages and reply by itself.
> >
> > The reason I suggested new messages was that I would solve that by
> > declaring that these new messages have whatever magic semantics I need
> > to make this work ;-)
> 
> ah right, the famous DeusExMachina message type then :-)

:-)

Ian