Message passing

[Jonathan Shapiro]

I have a question about message passing/RPC type things.  I'm hoping
some of you will be able to point me to papers.

I've been comparing the Mach and the KeyKOS kernel message interfaces,
and reading about the RPC work that Bryan et al have done at Utah.  It
strikes me that Mach and KeyKOS take two VERY different philosophical
approaches, and I'ld like to understand the relative merits.

KeyKOS:

KeyKOS restricts messages to a single page of contiguous data (not
necessarily page aligned), if shared memory is not appropriate, larger
blocks of data can be transferred in page-size chunks with a simple
application to application protocol.  As I see it, this design has three
principle recomendations:

1) The kernel path is very simple, being a contiguous byte copy that
can be performed without intermediate buffering or software
implemented scatter/gather.

2) Kernel objects do not need to worry about doing scatter/gather for
incoming messages, which makes it feasible to do object-specific
buffer treatment in several places that might otherwise be difficult.

3) A minimal (and fixed) number of system resources need to be pinned,
guaranteeing that it is feasible to transfer a message atomically.

Similar arguments apply to limiting the number of keys transferred,
though KeyKOS implements a sort of scatter/gather here by allowing the
application to name the "key registers" that keys should be
sent to/received from.

Mach:

[I'ld appreciate corrections and/or pointers to appropriate
documentation.]

If I understand things, Mach implements no upper bound on message data
size, and is prepared to do essentially arbitrary scatter/gather
logic, all implemented by the kernel.  Mach also has both by-value and
by-reference transfer mechanisms, and these appear to be handled in
the message system [would this be simplified if Mach had a system-wide
way of naming memory objects?].  It looks to me like a client app can
construct a descriptor for an arbitrarily complicated message,
potentially including overlapping buffers (semantics of this is
curious, and I can't find where it is specified).

If there really is an unbounded number of resources involved in a
message transfer, it strikes me that this opens us up to both
potential deadlock (which get turned into out of memory faults) and
denial of service attacks.

My terseness here reflects ignorance rather than prejudice.  

I see how to retrieve the kernel-object simplicity by having the
kernel conditionally copy a scattered message into a dedicated kernel
buffer to allow kernel objects a simple access mechanism.


I'm contemplating how messageing should be done in DIMSUM.  I'm very
impressed by the presentation composition work described in the
presint.ps paper Bryan has mentioned.  I'm inclined to think that
there should to be an upper bound on the amount of data that will be
atomically transferred.

I'm also aware that scatter/gather can be important.  I don't really
understand how this relates to IPC, and I'ld like to.  Is the
complexity of kernel-implemented scatter/gather really outweighed by
the advantage?

Anybody got a good example where scatter/gather messaging is a
significant improvement, or a pointer to a clear paper that discusses
this issue?



Jonathan