Re: /proc/xen/xenbus supports watch?

[harry <harry@hebutterworth.freeserve.co.uk>]

FWIW:

I think most of the ugliness here is a result of confusion from using
the store to do both the storage of persistent configuration information
and as a mechanism to implement configuration ABIs.

If you split out those two functions such that there is a persistent
store service which is generally useful for storage of persistent
configuration information but only stores information that is private to
each client then you can allow clients to lock down their private data
for unbounded periods (because it doesn't matter to any other client)
and there is no need for transaction retry in the clients.

You have to provide a separate mechanism to publish configuration ABIs.
The provider of a configuration ABI can sign up to a maximum service
time for requests made of that ABI.  If a request takes too long, it can
be promoted to a domain failure of the provider domain.

So a scenario for a misbehaving domain might go as follows: domain
misbehaves and accidentally locks up part of its private region of the
store.  This state can persist indefinitely without affecting other
domains. Super-user performs some configuration which results in a
request to the domain's configuration ABI to reconfigure something.
Domain attempts reconfiguration but hangs because it has locked up a bit
of its area of the store. Configuration request violates domain's config
ABI SLA.  Domain is killed.  Super-user's configuration request fails.
Store detects domain exit and rolls back uncommitted transaction.
Domain is restarted.  Super-user can retry configuration request.

With this approach, clients of the persistent store don't have to handle
transaction failures.  Clients of the configuration ABI have to handle
failures of configuration requests as a result of domains exiting (which
is unavoidable: failing requests could be automatically retried after a
domain is restarted but a domain might fail continually in which case
the configuration request would eventually have to be failed).

This note might not be very helpful for 3.0 given the current
architecture but you might perhaps find a mapping of the above solution
onto the xenstore infrastructure.

On Wed, 2005-09-21 at 10:39 +0100, Keir Fraser wrote:
> On 20 Sep 2005, at 12:01, Rusty Russell wrote:
> 
> > The only issue is that, in the case of migration, the new xenstored
> > won't know about any transaction currently in progress.  We can either
> > migration transactions (easy for clients), or return EAGAIN for the 
> > next
> > operation (easy for xenstored, sucks for clients).
> 
> Well, you know we already disagree very strongly on this.
> 
> Either we allow clients to lock down sections of the xenstore hierarchy 
> for unbounded periods of time (unacceptable since we do not trust all 
> clients) or we have to handle transaction failure in the clients. The 
> only exception to this I can think of is read-only transactions, where 
> you can take a read-only consistent snapshot of the store when the 
> transaction begins and guarantee eventual success (even here we may 
> want a timeout to avoid resource hogging). Apart from that, on failure 
> the client *has* to execute its transactional code again -- the values 
> it writes to the store may be dependent on values it reads as part of 
> the same transaction. If the transaction is failed because some of 
> those values it read are now stale (or might be stale, because a leased 
> lock was revoked), the transaction replay has to include re-execution 
> of the client code -- it cannot be hidden in the transactional API 
> (e.g., just replaying the transactional writes from the previous failed 
> attempt may be incorrect if those writes are based on stale reads from 
> the previous failed attempt).
> 
> This is the price for providing ACID guarantees (well, A, C and I at 
> least, and without atomicity, consistency and isolation you are not 
> implementing transactions as understood by every computer scientist).
> 
> If we have to make failure visible to clients anyway (and I'm sure we 
> do), it does at least greatly simplify things like xenstored restart 
> and migration. Transactions are simply failed.
> 
>   -- Keir
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>