[cap-talk] How desirable / feasible is a persistent OCAP language?

Stiegler, Marc D marc.d.stiegler at hp.com
Thu Jul 17 12:54:30 CDT 2008

I seem to be on the bleeding edge of using persistence in OCAP languages, having written persistent code with both E and Waterken. My observation from the field is, new insights are sorely needed to make programming with persistent languages a satisfactory experience.

E's persistence is just plain hard to use. Someone who is interested should go through the cap-talk and e-lang archives to see where we left this discussion when I finished my last piece of work in this area. I think we had some ideas to make it better, but none to make it good.

I am tempted to say that waterken's persistence is too easy to use, although that is not quite right. Waterken's orthogonal persistence runs quite behind the scenes and is utterly unstoppable -- your stuff gets persisted, and revives exactly where it left off, it is like a bulldozer ... Or perhaps like a Terminator. On a normal day this is really cool. But on a bad day -- and bad days occur too often to just write them off as anomalies -- this can be a serious problem.  Recovering from an error can be so difficult that you have to throw away the whole universe -- not just one computer's worth, but a whole distributed network's worth. One of the specific circumstances that is most dreadworthy is the software upgrade, which in a waterken world is necessarily a live system upgrade. Alan Karp and I recently started a pilot program with half a dozen users using a waterken application we have under development. I have not added a single significant feature since, and am terrified of the need to do so.

Truthfully, I do not know whether our application would be any easier to upgrade if we were using a nonpersistent language. Upgrading the software in a distributed system when there is any persistent state is just hard. But the problems with upgrade in an orthogonally persistent system like waterken are so different from the problems I have had historically, it hurts in places where it has not hurt before, and is therefore more noticeable. But, as an example of an issue, the persistence in waterken interacts harshly with the garbage collection system. All the garbage in a waterken system gets persisted too (well, not quite. But while the programmer does not have to write any code to use waterken's persistence, he must understand the persistence and garbage collection systems in some detail to not be grievously surprised). The manual mechanism for garbage collection requires breaking the program into large numbers of vats, and killing the vats that have collected too much garbage. For the early rapid prototyping phase, one strongly desires to use fewer larger vats. But if you do that, the architectural transform to smaller vats introduces some truly spectacular upgrade problems. Perhaps I personally have now gotten comfortable enough with waterken vats to do my prototypes with vats that are not only small but also collectible. And I have developed some reusable tools and patterns for future waterken developers (as yet undocumented and unpublished). But for the skeptical person thinking about making the transition from Java to Waterken, this is a substantial obstacle.

As I pointed out years ago in an earlier discussion of persistence, one of the nice things about the traditional relational dbms and its schema is, when you need to do an upgrade, the schema is a clean compact representation of the core of the upgrade problem. Its format makes it easier to wrap your head around all the issues at one time. You can write a separate program just to massage the dbms into shape for the next launch of the normal (revised) application. You can even tinker manually if you screw it up badly. The ability to tinker in the event of a screwup substantially reduces the fear factor associated with development, which accelerates development. Persistence in languages needs these kinds of recovery/survival/upgrade options. I think that is true even if it is not an OCAP language ... Although, on reflection, it is interesting to note that smalltalk was also a persistent language, and yet I never had as bad a reaction to it. Was it easier just because of the ability to tinker in smalltalk? Or was it because I was not writing distributed applications? Or both?


> -----Original Message-----
> From: cap-talk-bounces at mail.eros-os.org
> [mailto:cap-talk-bounces at mail.eros-os.org] On Behalf Of Mark Miller
> Sent: Thursday, July 17, 2008 7:33 AM
> To: rmeijer at xs4all.nl; General discussions concerning
> capability systems.
> Subject: Re: [cap-talk] How desirable / feasible is a
> persistent OCAP language?
> On Thu, Jul 17, 2008 at 3:33 AM, Rob Meijer
> <capibara at xs4all.nl> wrote:
>       I personally feel that real persistent processes,
> (especially when using
>       an OCAP language to create the code for these
> processes) would greatly
>       help in reducing the complexity of least authority
> design, but given the
>       fact that this subject to my knowledge has not been
> more than slightly
>       touched on this list, I have no way of knowing if this
> personal stand on
>       is one that has a general consensus, is by general
> consensus just
>       considered clueless, or has both lots of people
> agreeing and disagreeing.
> Hi Rob, my sense is that there is a general consensus that
> ocap languages should be made persistent. (This is of course
> a great opportunity for those who dissent from that consensus
> to speak up.)
> The previous discussions we've had on the list, and the
> remaining controversies, regard how to get there. Should
> persistence be orthogonal or not? Given either answer, how do
> you handle upgrade?
> <http://www.erights.org/data/serial/jhu-paper/upgrade.html>
> <http://waterken.sourceforge.net/upgrade/>
> Should it be built into the kernel/TCB (Smalltalk, Waterken,
> Newspeak? <http://bracha.org/Site/Newspeak.html>), or should
> it be written in the language with no undue privilege
> <http://wiki.erights.org/wiki/Safe_Serialization_Under_Mutual_
> Suspicion>?
>       Next to the desirability question, a second question
> might be just as
>       important, how feasible would it be to make one (or
> more) of the current
>       OCAP languages into a persistent language, and if its
> feasible, would the
>       private storage facility offered by MinorFs be
> sufficient to make making
>       that language persistent be sufficient to make image
> storage completely
>       adhere to POLA itself.
> The key constraint that Joe-E enforces, that enables Waterken
> to provide orthogonal persistence, is that Joe-E objects
> cannot advise their own serialization through the Java
> serialization mechanism. They can interfere with it only to
> the extent of preventing serialization. Should serialization
> of a Joe-E graph succeed, it's guaranteed orthogonal.
> --
> Text by me above is hereby placed in the public domain
> Cheers,
> --MarkM

More information about the cap-talk mailing list