[cap-talk] Union of runtimes (was: I Recant)

[Stiegler, Marc D]

> On Tue, 2007-01-16 at 16:20 -0800, Mark Miller wrote:
> > For distributed client-server applications, when the server is 
> > defensively consistent...
> 
> The phrase "when the server is defensively consistent" is 
> exactly the place where my reaction becomes "true but devoid 
> of practical content from the developer perspective" [note I 
> should have written "developer"
> earlier].
> 
> You are always careful to add the qualifying phrase, which I 
> greatly appreciate. My problem with it lies in the concern 
> that this qualifying phrase reduces the set of currently 
> interesting programs to a set of size zero. The mintmaker is 
> a nice example, but it is a toy example.

The term "toy example" can be misleading. The IOU protocol
implementation in E is also so small it might be called a "toy", yet it
is actually a production-quality electronic rights transfer system
except for extreme performance-dependant applications (E's performance
limitations in compute-bound applications is unimportant in most such
applications, network latency is the typical bottleneck). Small does not
mean toy. Indeed, for applications with harsh secure cooperation
requirements, big sprawling unreviewable systems are the toys that are
not production-quality. The IOU protocol mint is also defensively
consistent.

At this point we have a surprising number of defensively consistent
demonstrations. Habitats was a multi-million line virtual reality
written with defensive consistency pervasively incorporated. DonutLab,
while so small you might call it a toy, actually meets most of the goals
of PlanetLab, which no one could call a toy -- the reason DonutLab is
toy-sized is that we get so much leverage out of the E API for this
particular app that we are looking at a hundred-to-one reduction in code
requirement (this is *not* a claim of hundred-to-one advantage in
general, but this particular case enormously leveraged our advantages).
The toy-size makes it easy to do the review that demonstrates ubiquitous
defensive consistency (the IOU-based mint is one component in the
DonutLab). The DarpaBrowser demonstrated defensive consistency between a
web browser and a set of rendering plugins, a truly complicated boundary
to make defensively consistent. The sash powerbox in Emily demonstrates
defensive consistency in a safe-shell-command framework. The CrashBot
program, which is part of a paper I'll be presenting at the C5
conference in Kyoto next week, demonstrates defensive consistency in a
4-party game. The e-chat example in Walnut demonstrates defensive
consistency in a peer-to-peer chat tool which, even though it is only
two pages long, implements all the core features of a full chat system
and is once again so small only because we get such enormous leverage
from the E API -- Microsoft did a chat system with similar
functionality, but none of the defensive consistency, in C# as a 15 page
example. The security review demonstrating the defensive consistency of
echat is a little longer than echat itself, mostly because of the
discussion of ways in which it could have been made not defensively
consistent, the consequences, and techniques for avoidance. The
capsicomm web server, pathetically small and incomplete as it is,
nevertheless demonstrates defensive consistency between a web server and
its scripts/servlets/plugins. CapDesk demonstrates defensive consistency
between an gui application launching framework and its applications. It
also demonstrates defensive consistency between a set of file system
clients and a set of file system servers, sort of a
defensively-consistent FTP server system.

There is hardly a category of software development for which we do not
have at least some demonstration that, if examined closely, credibly
demonstrates that defensive consistency is compatible with the goals of
the application area.

> 
> On reflection, the issue that needs to get talked about here 
> is "Well, what is the impact of adopting defensive 
> consistency as a design model?
> How does this narrow the set of useful and needed 
> applications that we can build?" Note: I do not mean "how are 
> must the actions of these applications be narrowed". I mean: 
> "taking the real application requirements as immutable, which 
> applications can and cannot be built within the defensively 
> consistent model?"

What is the impact? This question is too enormous, in the general case,
to be answered in detail. But the consistent high-level truth that
emerges from all the examples I listed above is this: if you compare the
cost of developing almost any application using a popular language
(Java, C++) without defensive consistency, to the cost of developing the
same application with defensive consistency using a suitable,
well-crafted object-capability language (E, Emily), the object-cap
approach reduces development cost, reduces maintenance cost, increases
reliability, and decreases vulnerability. Of course, I am cheating here:
popular languages are pathetic compared to the best we can do in
languages that do not support defensive consistency. Comparing Emily to
Ocaml, or Java to Joe-E, the comparison data is limited but nevertheless
strongly suggestive that the change in programmer behavior, even the
change in system architecture, is modest. Of the interminable examples I
gave above, only three had components for which interestingly different
code had to be written to control the trust boundaries: Habitats,
CapDesk and DarpaBrowser. In CapDesk and DarpaBrowser, even this special
code was a small part of the cost of the system as a whole (in Habitats
the answer is the approximately the same, though complexified by the
extent to which we were learning how to build the defensive-consistency
tools at the same time that we were implementing the system; doing it
over again with today's tools, the answer would be the same as with
CapDesk). In echat, sash, CrashBot, capsicomm, and DonutLab, almost all
of the defensive consistency came about as an emergent property of
ordinary modular OO design: i.e., the programmers didn't have to do
anything except use an appropriate language in a traditional way.

> 
> Also, what would be the cost of rebuilding these applications 
> where needed, and more importantly, do we understand enough 
> about the construction idioms to start teaching some of them 
> to programmers?

Rebuilding the apps is going to be expensive, no doubt about it. It will
be much less expensive to rebuild than it was to develop the apps from
scratch, both because you have a great spec to start with (the original
app) and because the tools yield significant productivity improvements
(not the factor of one hundred like Donutlab, but possibly a factor of 3
or 4). But it is still expensive -- unless you compare it to the cost of
constantly patching the broken, existing systems, in comparison to which
the rewrite cost is so small it is almost negligible (alas, a negligible
cost of a couple hundred million dollars is quite real, unless you
actually somehow wrap your hands around the $20B wasted trying to heal
the dead, which is what the computer security industry has been doing
every year for the last 6 years).

Can we teach the idioms? To some extent we already are, my personal
contributions to this being the patterns section of Walnut and the
PictureBook of Secure Cooperation. To some extent we don't even have to
teach the idioms for them to be used: Ocaml programmers moving to Emily
will immediately get defensive consistency benefits for little more than
the cost of learning how to turn on the compiler and interface to an
appropriate powerbox, and the same is true for Java programmers using
Joe-E.

--marcs