[cap-talk] Claim: correct generic wrapping is not possible in principle

[Jonathan S. Shapiro]

On Mon, 2007-01-01 at 23:30 -0800, Jed Donnelley wrote:
> At 09:15 AM 1/1/2007, Jonathan S. Shapiro wrote:
> >On Sun, 2006-12-31 at 19:58 -0800, Jed Donnelley wrote:
> >
> > > A general solution seems simple to me - namely partial buffering
> > > of messages, both for data and for capabilities.  This immediately
> > > occurs to me because it was the approach used in NLTSS (though
> > > there only for data as capabilities were data).
> >
> >I suspect that you are once again thinking first about the network case.
> >Yes?
> >
> >In the local case:
> >
> >    Partial buffering with dynamic bound leads to unpredictable blocking
> >    behavior and kernel resource exhaustion.
> 
> I don't agree.  I have detailed experience with the NLTSS system
> in this regard (only data buffering, not capability buffering, but
> I don't think the distinction is relevant).  I'm sure you agree
> that the buffers in the sending and receiving process ("user" level
> buffers) are necessary....

I do not, but this statement (and your later comments in the same note)
clarifies why: we have a disconnect of terminology. I would not refer to
the user-level source area or the user-level destination area as
"buffers".

The IPC literature has come to use the term "buffer" to refer
exclusively to *additional* storage that is used to make a temporary
copy within the operating system kernel. Data transfers that proceed
directly by copy from sender memory to receiver memory are said to be
unbuffered. In the context of IPC, this use of the term "buffer" has
prevailed in the literature since the late 1980's.

So: my comments about buffering refer only to additional buffers within
the kernel. I am stating that such buffers should not exist. From your
note you seem to agree (! :-).


The rest of your comments about message length seem to concern the
distinction between bounded and unbounded payloads (e.g. your comments
about "packetized" transfers).

Several years ago Charlie Landau and I convinced ourselves that we knew
how to get unbounded transfers "right" in the kernel sense. It requires
more IPC-related process states, and the implementation is a nuisance,
but it can be done.

My objection to unbounded messages in the local case are not based on
any issue that arises in the kernel implementation (which I refer to as
the "transport layer"). My objection has to do with what happens at the
application layer when a receiver must be prepared to accept unbounded
messages. There are two cases to consider:

1. Receiver does not trust sender, because sender may send large
messages as a means of executing a denial of resource. This is readily
addressed by specifying a receiver-side limit on message length, but
doing so limits the feasible set of protocols.

Also, this approach tends to lead to multithreaded receivers, because a
receiver can remain blocked to other callers for a long time under
conditions of severe memory pressure. Multithreaded services are, in
turn, less reliable.

2. The sender may not trust the receiver's user-level page fault
handler, but the receiver is unable to allocate unbounded storage. This
is a particular problem when the sender is a service returning a result
having dynamic length to a (presumptively untrusted) client. In this
situation there appear to be only four possible resolutions:

  a) Require that all page fault handlers be trusted. [By page fault
     handler, I mean the process that decides what pages to bind
     where and handles permissions faults. Paging to and from the swap
     area is a separate issue, and is not a problem here.]

  b) Let the sender block while the receiver allocates. This is the
     commonly used answer, but it introduces the possibility of denial
     of service/resource attacks against the service.

     The usual "fix" is a server-side timeout. This merely serves to
     make the system's behavior unpredictable under load, because
     timeout error conditions of this type are extremely rare and
     universally untested in practice.

  c) Truncate the message.

  d) Transfer the message indirectly. This has to be done with care
     to ensure that denial of resource cannot occur. This is the
     purpose of the TBO in my paper.

shap

-- 
Jonathan S. Shapiro, Ph.D.
Managing Director
The EROS Group, LLC
+1 443 927 1719 x5100