[cap-talk] Memory access based OS security

[Bill Frantz]

I think both Shap and Ben misunderstood what I was trying to say. Let me
try again.

We are comparing two ways of isolating the effects of one program from
another. In one case, the language based case (LB) we depend on "memory
safety" to keep the programs separate. In the other, the traditional OS
case (TOS) we depend on hardware memory protection. (Yes, I know there are
other techniques, such as the GE 235/Datanet 30 system at Dartmouth, but
they haven't been used for 30 years.)

In terms of assurance we can use a broad range of techniques from fixing
bugs as they to occur thru formal methods. Regardless of assurance
technique, the size of the code that needs to be checked has a strong
effect on the difficulty of the checking.

Given that we are talking about operating system, and not applications, we
need to provide a mechanism for untrusted/hostile users to run new programs
in the system. LB systems limit the range of codes that can be run to those
which are "memory safe", while TOS systems allow any sequence of machine
instructions.

With TOS systems, we need some assurance for the code of the TCB and the
correct functioning of the hardware. In KeyKOS, CapROS, and VM/370; the
systems I am most familiar with, this code is mostly involved with
providing basic services like message passing (the first two), virtual
machine emulation (VM/370), and virtual memory (all three). All three of
these systems were written in low level languages (VM/370 and KeyKOS were
written in 370 assembler while CapROS was written in C and KeyKOS was
re-written in C). Verification consisted of code review and bug fixes.
KeyKOS and VM/370 demonstrated a high degree of robustness while CapROS is
too new to have a track record.

I am less familiar with LB systems. Java depends on a byte code verifier
and Smalltalk depends on a compiler. In both cases, we have a additional
piece of code which needs to be verified. Verification in these systems
seems to be also via code review and bug fixes, with more formal techniques
still in the research stage. Both of these system have a track record. Java
has proven itself in server-side applications. Smalltalk doesn't seem to be
used in production applications, but it has proven reliable for prototyping
and ad hoc programming.


Both systems have advantages and disadvantages: LB systems can provide a
much more efficient security domain switch. TOS systems can support a wider
range of languages and provide reasonably efficient virtual machines. Both
systems could be implemented in memory safe languages, although the TOS
systems I am familiar with have not been. Systems which combine the
approaches could be quite interesting, although I have not heard of any
being designed.

Cheers - Bill

---------------------------------------------------------------------------
Bill Frantz        |"After all, if the conventional wisdom was working, the
408-356-8506       | rate of systems being compromised would be going down,
www.periwinkle.com | wouldn't it?" -- Marcus Ranum