This note describes a significant reformulation of the EROS capability invocation path. In particular, it describes how the difficult corner conditions are handled without impeding the main code.

1. Problem Statement

The EROS/KeyKOS/GNOSIS capability invocation path semantics is complex (perhaps too complex). Here are some of the semantic complications:

• Invoking a process capability with a SEND operation causes a new thread of control to be created.

• Performing a RETURN on a kernel object capability with a resume capability rsm(xx) in slot 3 causes control to be returned to process xx rather than to you. This is in marked contrast to "system call" style interfaces, where control returns to the caller eventually (though POSIX fork/exec are similar).

• Performing a RETURN on a kernel object capability without a resume capability in slot 3 retires the invoking thread of control unless it proves that we have invoked a kept red segment key.

• Performing a SEND on a kernel object where capability argument 3 is not a resume capability causes the return value(s) returned by the kernel object to be made to vanish, even though the invoker continues to execute (SEND returns no arguments).

• Performing a SEND on a kernel object where capability argument 3 is a resume capability rsm(xx) has the return values delivered to xx, but also creates a new thread of control for the invoker.

• Performing any invocation on a red segment key may invoke it's keeper, but unlike the previous cases the fact that control is returning to another process cannot be immediately discerned from the arguments presented at the call.

• A few kernel objects permute or swap capabilities. If the invoker and the invokee are the same then juggling the capabilities can become very interesting.

• A few kernel operations can alter the runnability of a process -- in particular can alter the runnability of the invokee. This means that a perfectly good invokee may no longer be runnable by the time we are ready to transfer control to it.

• If the invokee was named by a fault capability, all result value(s) of the operation should be discarded. More precisely, they should be treated as though they are a fault capability invocation, which has "distinctive" semantics.

This note describes how these corner cases are handled by the kernel.

2. Synopsis of Solution

All of these cases can be reduced to a couple of bits of knowledge a few special cases:

• What capability is being invoked?

• Which process (if any) will receive control when the invocation is done?

• To which process (if any) should results go?

• Should the invoker continue execution (i.e. is this a SEND operation)?

• Is the present operation one of the few that might mangle the invokee? If so, this is dealt with by the code that implements the operation. Handling this is straightforward, so we will ignore this issue from here on.

In older implementations of EROS, all of this was managed by ensuring that results went to a temporary holding buffer. A boolean was tracked indicating whether transfer of these results to the final recipient should be suppressed (to handle fault capabilities), and the output process could be nullified, signifying either that it had ceased to be runnable or that the invocation was a SEND on a kernel object.

This implementation has several problems:

1. Molasses in January is faster.

2. Results are copied even when it isn't necessary. It proves not to be necessary most of the time.

3. A boatload of complexity is introduced into various pieces of code to see that this side structure -- which can hold capabilities -- is properly handled during consistency checking and key ring management.

4. The design necessitates setting up the side buffer in the common path.

A conversation with Norm at one point suggested an extremely elegant alternative: use a shadow process. In this design, the "context" of an invocation tracks four pieces of information:

struct Invocation {
  Key     *key;     // the key invoked
  Process *invoker; // the invoking process
  Process *invokee; // the receiver of control
  Process *target;  // the receiver of results
};
	      

An entry in the process cache (actually, one per processor on an SMP) is reserved to serve as a "shadow" process. The key and invoker fields are set in the obvious way. The invokee and target are initially set according to the invocation type:

The fault key cases are unimplemented primarily because I cannot really see a situation in which the results of a kernel object invocation can rationally be directed to a fault key. A corollary of this is that the returner also should not permit this.

If it is subsequently discovered that this was an invocation of a kept red segment key, then the key field is updated to name the gate key to the keeper, and both invokee and target are revised to name the process named by the gate key.

3. Invocation Processing

Given section (2) as preface, it is now possible to describe how invocation proceeds.

1. The invocation trap is taken and the process registers are saved. We switch to the kernel stack.

2. The capability invocation arguments are syntax checked. This involves bounds checking the capability entry/exit vectors and verifying that the invocation type falls in the range [0..2]. We also do an upper bound check on the entry string, but not on the exit string, as the upper bound on that is implied by the presence of an upper bound on the entry string.

3. The invoked capability is determined. The capability type byte is extracted. The invocation type is OR'd into the resulting value. We now have an index into a table of the form:

   captype 0:  { return_op, call_op, send_op, unused }
   		  

4. We set the invoker state to waiting in the optimistic hope that this is a CALL operation, write the invocation context structure as described for the CALL operation, and dispatch through the table.

5. If it is not a call operation, we will end up in the generic return handler or the generic send handler. These are wrappers that reset the invoker state, examine the capability in slot 3, conditionally reset the invokee and target, and then call the handler specified in the CALL slot for that capability.

6. If the invocation was not a CALL, control eventually returns to the SEND or RETURN wrapper. The SEND wrapper fabricates a new thread for the invoker if necessary. Both wrappers return to the main path.

7. The main path now checks if invokee == 0, if so, it branches to the Yield entry stub.

8. Control now returns to invokee.