[cap-talk] Another "core" principle - virtualizing memory

[Jed Donnelley]

At 08:59 AM 1/1/2007, Jonathan S. Shapiro wrote:
>On Sun, 2006-12-31 at 20:15 -0800, Jed Donnelley wrote:
> > >If we restrict ourselves to capabilities implemented by processes
> > >(presumably stretching to include the kernel as a distinguished
> > >process), we cannot account for address spaces, memory mapping, or
> > >shared memory within the protection model of the system. That seems
> > >disastrously bad.
> >
> > I don't believe any of the above is true.  I believe one can
> > use data (file, memory, you pick a name) objects and pass then
> > to Process objects for mapping into memory...
>
>You have a reductio problem buried in your response, because it is not
>possible to pass memory. It is only possible to pass a mapping of
>memory, and we must not make our solution rely on that because the
>problem we are discussing is how to construct such a mapping within the
>object-capability model.  Ditto files (fundamentally, a file is built on
>an address space).
>
>It is the "mapping into memory" part that we cannot account for. How,
>using only object-capability mechanisms, does the process do that, using
>only object-capability mechanisms, without making the primitive storage
>objects (pages) and the mapping tables (page tables, but in KeyKOS/EROS:
>Nodes) first class objects that are named by and manipulated through
>capabilities? [I concede that there are other possible data structures
>than Nodes, but there will be *some* data structure that records the
>relationship between addresses, permissions, and pages]

By hiding that processor specific part of a process within the Process
object, where it belongs.

>...
>Jed: if this is part of the Process object, please explain how processes
>can share memory,

This is what I described before that you criticized as merely a mechanism
for distributed shared memory.  It can be distributed or not, it works
locally or remotely.  It works because this is how processors must work.
Just to repeat the overview (for anybody that didn't read the other
message in detail), the basic idea is that data storage objects
(call them files, or segments, or whatever) are passed to the Process
object for mapping into memory.  They need appropriate locking with
notification (write locks and read/write locks) for the shared
memory application.  This is all rather simple.

>and please give a sane story for storage accounting if
>(1) the space used for the mapping is accounted to the allocating
>process, the space is (partially or wholely) shared, and the allocating
>process exits, OR (2) two processes interleave mapping operations within
>a shared region.

The metadata for the process is "charged" to the process.  The
memory when it is used (mapped and active) by a process is charged
to the process.  The data storage is charged to the "file" object,
which also is charged for it's own metadata of course.

>No. The decision to make address spaces first class independent of
>objects, and the decision to make them recursively defined with pages as
>a (degenerate) atomic address space, were two of the most elegant and
>beautiful decisions of GNOSIS. They constitute the first successful
>conceptual disentangling of address space design confusions that go back
>into the earliest multiprocess operating systems (or at least, the first
>that I know about).

Sorry, you lost me in the above.  I presented a solution to a problem
that you defined.  Do you believe it isn't a solution?  If so, why not?

> > Hmmm.  I'm not quite sure where this discussion stands.  All I care
> > about is that the objects needed for a practical (well performing,
> > subject to any needed sort of analysis, etc., not necessarily up to
> > some arbitrary standard of "elegance", which may be a matter of taste)
> > systems can be wrapped as needed for modularizing higher level
> > services (e.g. remote access, revocation, responsible delegation,
> > etc.).
>
>We haven't lost that. We've merely explained why your belief that it was
>trivial to wrap objects denoting real memory was wrong. In a nutshell,
>you were busy considering the network distribution problem, and failed
>to fully consider the requirements of front-ending page capabilities in
>the local case.

I agree that it's possible to design primitive (kernel) objects in
systems so that virtualization isn't possible.  RATS did that, so
that much is obvious to me.  However, I also argue that it's possible
(in fact very simply so) to design a system that includes mapped
virtual memory in which all kernel objects are virtualizable.
Not only that, but I argue that it can be done efficiently.  I
believe this because I've seen all the details laid out.  I believe
doing so is pretty much independent of processor architecture,
as long as the mechanism is contained within the appropriate objects.

About the accounting I'm not so sure, but I consider that a
simple problem.  I admit I consider it a problem that doesn't
need a perfect solution to work well.

--Jed  http://www.webstart.com/jed-signature.html