[cap-talk] Persistence as a cap value (was: Re: ...PLASH discussion)

[Jonathan S. Shapiro]

On Tue, 2008-03-11 at 23:27 -0700, Jed Donnelley wrote:
> The fact that persistence issues keep surprising (me anyway,
> others?) suggests to me that perhaps this major classification
> category for capability system may not be given enough emphasis/
> clarity.  I believe that in my heart I always think of capability
> systems as manipulating persistent capabilities and keep being
> surprised when I find that many (most?) aren't.

Well it won't surprise you that (a) I agree, and (b) I've put some
thought into this.

Terms: explicit persistence vs. orthogonal persistence. The orthogonal
kind is transparent.

It comes down to two issues: instances and consistency. Here are what
seem to be the key intuitions that I have come to.

I. NAMING AND PROTECTION

In any system, *some* things are persistent. Any persistent thing can be
induced to have a durable unique identity, and given a durable unique
identity we can both name that thing by a capability (it can be a
target) and we can associate capabilities with that thing (it can act as
a capability container).

In explicitly persistent systems, application images are generally
persistent, and it is easy to say things like "any instance of
application A should be granted capabilities X and Y". Process instances
generally are NOT persistent, so it is not straightforward to associate
capabilities with process instances in a way that will survive shutdown
and restart.

The nearest approximation to this that I have found is something like
the following:

  1. Invent a "process instance ID allocator" that allocates unique IDs
     for process instances, and associates each such ID with a binary
     image and a schedule.

  2. Provide means for an application to store associations between an
     application ID and capabilities. Further, provide a means for
     voluntary per-process persistence.

  3. On re-start, kick off new copies of those processes under the old
     IDs, and let them unpickle the old state to the extent that they
     want to.

The problem is that this really only seems to work out sensibly for
things like daemons. You can get the processes restarted okay, but you
then need to re-build all of the *connections* associated with those
processes, and that is hard. You're almost forced into a sessionful
communication paradigm (as opposed to RPC) so that the various parties
can re-negotiate consistency.

In the end, I suspect that the complexity of this is fairly large, but I
haven't actually tried to implement it.

In practice, however, it would all seem to depend rather heavily on how
many times you have instance-specific capabilities that need to survive
shutdown.

II. CONSISTENCY

The BIG win of orthogonal persistence -- and also the big loss -- is
that all processes within a persistence domain come back in mutual
agreement about their state. This seems to be pretty important if you
want to do domain-based decomposition, because you don't want to have to
re-establish consistency *internally* within each application; that
would add quite a lot of complexity.

But once again, it depends on whether process recovery is important for
non-daemon processes.

And an argument can be made that process persistence is the wrong
default. To the extent that programs are known to be buggy, it may
actually be important to restart them periodically purely to clear out
accumulated muck. Persistence adds a lot of pressure for correctness,
and human programmers don't seem to be very good at that.


shap