[cap-talk] Capability levels

[Jed at Webstart]

At 06:18 PM 8/14/2006, David Hopwood wrote:

>...the layering is "hardware \ language1 \ OS \ language2 \ application",
>where "\" means "is below". From an application programmer's point of
>view, and from the point of view of OS design rather than implementation,
>however, "language1 \ OS" can be viewed as a single layer.
>
>(The OS may be implemented in more than one language. Also, some of what
>appears as "the OS" to application programmers may in fact be implemented
>in per-process libraries, rather than in the kernel. But these are just
>implementation details.)

I believe this difference in views arises from a difference in 
'operating system'
architectures.  To me the above suggests the typical monolithic operating
system architecture - e.g. Windows or Unix.  My thinking about operating
systems reflects what's typically referred to as a micro kernel architecture
where all processes (active objects) are in a sense equal except in so far
as some may have more permissions than others (POLA).  E.g. consider
the NLTSS architecture as depicted in:

http://www.webstart.com/jed/papers/Components/Figure-4-50.gif
(from http://www.webstart.com/jed/papers/Components/ )

In such an architecture, what's referred to as the "message system"
(and at least the network driver, perhaps the other drivers in terms
of the implementation) seems to fit into the sort of 'operating system'
category that you've depicted in your diagram above.  This part of the
"system" supports only the "invoke" or "communicate" functionality
of the system.  All services that are typically considered operating
system services (e.g. file access, process creation and manipulation,
directory support, user/account management, etc., etc.) are made
available through processes with more or less privileges.

I believe that many/most micro kernel architectures are comparable.
For example, one could consider Mach and it's "port"s.  The
micro kernel implements the communication primitives on ports,
and user mode processes (with various permissions needed to
get their jobs done) implement the majority of the services that
are typically considered to compose the "operating system".
All such processes are given whatever privileges (permissions)
they need in terms of ports to perform their needed services.
<unfortunately the view of the Mach kernel is often hidden
under a set of apparently monolithic "OS" services as in MacOS>.

Considered in the light of such micro kernel architectures, what do
the "Capability Levels" look like?  I believe that in such architectures
any software that goes into the micro kernel can more appropriately
be considered as part of the hardware level - e.g. like in a machine
with capability hardware such as the Plessey-250.  All it provides
is an invoke or communicate "instruction".

What are typically considered application level processes (e.g.
those initialized by users like cat or cc or ...) are at the same
"level" as the operating system processes such as the file
server, process server, etc. - only being distinguished in that
each has permissions appropriate to the service it needs to
provide.

With this architecture the capability levels that make most sense
to me might be diagramed like (along the lines of figure 4 above):

    Alice (app)             Bob (app)                Carol (server)

network level cap   <n> network level cap    <n>        network level cap
IPC level cap          <i>      IPC level cap           <i>     IPC level cap
language level cap <m> language level cap <m>   language level cap

where:

<n> denotes network level communication,
<i>  denotes inter process communication, and
<m> denotes shared memory communication within a "process"
         by language level convention(s).

The fact that Carol might be providing a service that's considered
part of the "operating system" is of little consequence with regard
to levels or even really of "control".

What I've been referring to in the past as an "operating system"
level capability is really for me essentially an IPC level capability.
That it, it's implemented at and is a convention at the inter process
communication level in a system.  It would be delightful if in some
sense (implementation, standard, etc.) this same capability
(permission token) at the IPC (operating system) level could be
communicated (delegated) and used (invoked) at the network level -
but sometimes (sadly) this doesn't seem to be possible.

Similarly when one considers language level capabilities.  Such
capabilities can be implemented using shared memory structures
within a "process" (active object - though here perhaps there is
a meaningful distinction?).  As with any other sort of language level
representation (e.g. floating point numbers, strings, method
entries, etc.) the language determines the conventions and uses
for such capabilities - perhaps using shared libraries.  Such capabilities
may or may not be useable and or communicable across IPC level
communication.

As I hope you can see, for me the "level" of the capability
connotes how widely it can be used:

I.    Language level capabilities can nominally only be used within
a single language, or perhaps by conventions between languages
inside a single process (shared memory, OS level protected domain)
though shared library conventions.

II.   IPC (operating system) level capabilities can be used
through inner process communication between processes
(e.g. run by different "users", in different protection domains,
etc.) within a single "operating system" (think DVH, KeyKOS,
or EROS) which defines the capability invocation and delegation
mechanisms between processes.

III.  Network level capabilities can be used by any process anywhere
on the network.  They are generally a higher level abstraction that
can be used by processes of many different types (e.g. with
or without confinement, nominally capability systems or
ambient authority systems, etc.).

Perhaps after the above discussion you can see how the issue
of latency crept into my discussion.  While I agree that latency
is an implementation issue, generally:

1.  At the language level there is no need to do domain changes
(in the OS sense, register sets, virtual memory maps, etc.) to
invoke or communicate capabilities, so the latencies can be
kept down to the level or machine instructions, or at least to
that of subroutine entries.

2.  At the OS (IPC) level there is a need to do a domain change
to invoke or communicate capabilities - in most cases requiring
more mechanism and inducing more latency - often on the
order of 10x or more.  Here register sets possibly including
memory maps need to be exchanged.  The latencies here are
on the order of "exchange" times - typically some 10x or more
above subroutine times.

3.  At the network level we're talking about messages communicated
through packets across a network, typically with many milliseconds
or more of latency, typically some 10x or more above IPC exchange times.

A micro kernel system like Mach, uses (as I recall from memory) some
shared memory mechanisms to cut down on the latency of communication
through ports between processes at the IPC level.  This is fine as
an effort at optimization, but there is still an inevitable cost to
doing a domain change between processes.  There may also be
a cost for such optimizations in terms of their extendibility (e.g.
to the network level) - in that such shared memory access may
be difficult to map to the network level.

> > ...I guess you're basing your argument on the issue of who's in
> > control?
>
>Yes. Isn't that the most important point?

As I hope you can see in the above - no, I don't think the issue
of control is the most relevant.  In a POLA architecture there is
no control hierarchy, but rather each independent entity is
trusted with just its needed POLA.  What's important to me
is the levels of communication that reflect the level of composition.

I see IPC level capabilities generally as being composed of language
level capabilities and network capabilities being composed of IPC
level capabilities - though I admit that independent capability
models (standards/implementations) can be created at all three
levels.  I generally see language level capabilities as needing
an extension mechanism to be used at the IPC level.  I generally
see IPC level capabilities needing an extension mechanism
to be used at the network level (DCCS was such an extension
mechanism).

My dream (I have a dream!!) [goal] is to blend, merge, combine
these levels, at least the terminology and I hope the implementations
(though perhaps with caching optimizations at each level) so that
we have a single capability (communicable permission token)
notion that works across all levels:

(hardware - if any), language, IPC, and network.

>...
>Can we agree on
>
>   1. Hardware
>   2. System language
>   3. OS
>   4. Application language
>   5. Network
>
>?

As I hope you can see from the above, for me a language is a
language and OS and application level services exist at the
same "level".  Capabilities can be conventions that are communicated
within a shared memory protected domain (whether part of an
operating system or some application) at what I consider the
"language" level or across such protected domains within an
IPC system (what I consider the "operating system" level)
or across a network.  So for me the levels come down to:

1.  Hardware (if any domain separation, supported by hardware)
2.  Language (any separation of domains supported by shared libraries)
2.  IPC (any separation of domains supported by IPC mechanism)
3.  Network (separate domains on independent systems on the network)

What do you think?  Is this just a difference of perspective (I'm
looking at it one way, you another, but it's the same thing) or is
there something fundamental about these distinctions and the
"level"s.?

Interesting discussion.

--Jed http://www.webstart.com/jed/