Run Time Environment

If you are trying to write real programs, it's useful to know what the state of the world is when the program begins excution. This page provides a description of how the usual EROS runtime mechanism works. The EROS kernel is not responsible for process construction; it is therefore not a party to any startup conventions about these programs. It is possible to build many different startup conventions. This is useful primarily for the sake of environment emulation.

The description given here describes the startup and shutdown conventions for native EROS programs. More specifically, this page describes the convention that is assumed by the EROS Constructor. There are a few primordial processes that implement these conventions a bit differently.

Briefly, the programmer's view of process startup is fairly straightforward. When the program's main() procedure is entered, the program has a well-defined set of initial capabilities in it's capability registers. The program performs any necessary initialization and becomes available for requests. At some later time, the process will be told to destroy itself, and will return a final result code. The following notes walk through the initialization and teardown of processes.

1. Process Fabrication

EROS programs are usually built by constructors, and the construction process if typically recursive. In the common case, even the process address space is constructed by another constructor. This leads to apparently conflicting objectives:

• The constructor cannot recursively invoke these constructors, because their runtime is potentially long, and it is undesirable for the process's constructor to be busy for long periods of time (principle: make the customer pay for it).

• The program cannot initialize itself without an address space.

To resolve this conflict, process fabrication is done by a temporary program known as the protospace from within the new process itself.

To minimize confusion, imagine that the program we are trying to fabricate will be called foo, and the address space for this program will be called foospace.

1.1 Construction

The first stage is to construct the shell of the new process and populate it with the protospace program. This proceeds as follows:

1. The foo constructor buys a new process from it's process creator. This process is unpopulated, so it's really just the framework for a process.

2. The foo constructor populates the key registers of this nascent process as follows:

   KR0	A void key. Key register zero always holds a void key. This value is hard-wired, and cannot be changed by the program.
   KR1	A read-only node key to the process's initial services (i.e. to the constituents node).
   KR2	A process key to the process itself.
   KR3	A start key to the process's process creator.
   KR4	A start key to the process's space bank.
   KR5	A duplicate of the process's schedule key.
   KR6	A constructor key to the constructor for the process's keeper.
   KR7	A constructor key to the constructor for the process's address space, or the address space key itself.
   KR8	A number key containing the initial program counter for the foo program. FIX: Is this still true?
   KR9..KR30	Void keys.
   KR31	A resume key to the process that caused this process to be started.

   This initial process image runs under the schedule designated by the requesting process. It's address space is a read-only page key to an immutable address space known as the protospace.

   The initial process has no keeper.

3. Control is now transferred to the protospace program. Protospace is most reasonably thought of as part of the constructor logic. The proper protospace entry point is well known to the constructor.

1.2 Protospace Initialization

Protospace must now construct the initial address space and keeper for the new process:

1. Protospace Fabricates the program's address space segment by invoking the address space constructor key (in this case the foospace constructor).

2. Protospace then overwrites the keys in slots KR7..KR8 with zero number keys; these keys will not be passed to the foo program unless they are included in the initial services node.

3. Finally, the protospace invokes the process key with the "swap space and PC" invocation, setting the address space of the process to the real address space (the foo address space) and the real entry point (the foo entry point).

4. Execution of the real program begins.

Note that protospace makes use of key registers during startup. When building an image map, programmers should not place capabilities that the program will need into capability registers. They should go in the constituents node.

1.3 New Process Startup

On entry, the new process inherits the key register state that was left by the protospace. This is identical to the initial key registers described above, except that register KR7 and KR8 hold zero number keys.

Depending on the type of address space used for this process, the process may initially have no stack or modifiable data. Three kinds of spaces exist in the current system:

• The address space is immutable, in which case having no stack is appropriate. A small number of primordial programs run from an immutable address space.

• The address space is a small space (i.e. one that does not have a keeper). In this case the space is initially immutable, but the runtime startup code immediately constructs a mutable stack. The process destruction logic assumes that a mutable space is 32 pages or less, with the stack at the top. Small spaces are provided primarily as a convenience for small utility domains, and are not suited to the construction of large service programs.

• The address space is a virtual-copy space, in which case the stack will be automatically filled in as page faults occur.

The run-time library determines the type of the address space by examining the global variable __rt_stack_pages. Details about how this is handled are described in Section 3. Once the stack and heap are allocated, the run-time library branches to __rt_start(), which is intended to give the high-level language runtime a chance to perform further initialization before invoking main().

If necessary, the process must arrange to call on it's own behalf any constructors whose products it may need.

2. Process Teardown

Just as the process must arrange to initialize itself, it must arrange to destroy itself if it honors the OC_Destroy request. Since the process created it's own address space, it must destroy it. careful look at the object reference manual will show that the destroy operation is expected to return. The question is, ``How can a program that no longer has an address space return to anyone?''

2.1 Process Finalization

A process that wishes to honor the OC_Destroy order code must hang on to it's process creator key. If the process runs in a kept address space, it must also retain access to the protospace page.

When asked to destroy itself, the process should first destroy (recursively) any services it has built. It should then arrange to jump back into the protospace address space for the last phase of the destruction by calling DestroyMyself().

On entry to DestroyMyself(), the process key registers should hold the following values:

How shutdown is accomplished depends on the process address space:

The DestroyMyself() library routine first sells back any process-allocated stack space, and then examines the process address space key to decide how shutdown should be performed:

• If the address space is immutable, or if the address space consists of a single page, the process can simply invoke it's process creator with the DestroyCallerAndReturn order code.

• If the address space is a more complex space, the process must first switch to a teardown address space (usually just protospace). The teardown space issues an OC_Destroy request on the process's address space and then invokes the DestroyCallerAndReturn request on the process creator.

The process creator now gets control, and demolishes the process structure. Finally, it returns to the designated resume key, confirming to the original caller (the one who asked the foo process to self-destruct) that the foo process has been successfully destroyed.

Note that protospace itself is a shared resource that does not get destroyed.

2.2 Caveats

There are some circumstances under which the destroy request may fail:

• The space bank from which the process was purchased has been destroyed.

• The process creator from which the process was purchased has been destroyed (this happens when the associated constructor is destroyed).

• The process itself has altered the state needed by the destruction logic.

Destruction is therefore done on a ``best effort'' basis. A client who wishes to unconditionally ensure that it's space can be reclaimed should arrange to fabricate its services under a separate subspace bank, and destroy the entire subspace. Note that doing so does not permit the services to perform finalization, and should be viewed as a way of recovering from buggy or pernicious services rather than as a way of tearing down programs in general.

3. Small and Normal Process Models

There are two primary runtime models for EROS processes: normal and ``small''. All EROS platforms support both process models. The normal model provides a fairly complete ``cocoon'' for the process. This relieves the application designer of the need to worry about page fault handling or stack/heap collision, but it comes at a cost: the application is required to have a memory keeper that will service page faults for the heap and the stack.

The small model exists to provide a low-overhead execution environment for processes that have only a small amount of state, and also to provide a minimal-demand runtime environment that can be used to implement the richer environment. Low overhead comes at the cost of correspondingly reduced runtime support.

Surprisingly, we find that many processes run very happily using the small runtime model. Where feasible, use of the small model is preferred, because small model processes have lower startup overheads. On some architectures, small processes also provide more efficient entry/exit costs during interprocess communication (start and resume key invocation).

As of this writing, the differences between the two models come down to two issues:

• Memory Allocation: How much underlying runtime support is assumed by the memory management libraries. In particular, who allocates dynamic storage?

• Small Spaces What address range restrictions are imposed on the process if it is to gain the benefit of faster context switch times?

At the implementation level, these issues are entirely independent: one concerns the behavior of the application runtime library. The other concerns favoritism applied to certain procsses by the kernel. Typically, however, we find that processes are concerned about both or neither of these issues.

Since the small space issue is simpler, we will deal with that first.

3.1 Small Address Spaces

On some architectures, notably the x86, the kernel provides preferred support for processes that use only a small amount of virtual address space. Specifically, any process that restricts itself to virtual addresses in the range [0, 32*PageSize-1] will be mapped by the kernel into a so-called small space. Small spaces take advantage of some dirty segmenting tricks to be simultaneously mapped in all address spaces, which leads to significantly improved context switch efficiency. Programmers writing performance-critical code may wish to take advantage of this by writing programs that meet the small space constraint.

Under the covers, the kernel actually runs every process as a small process until the first time the process violates the small process addressing constraints. That is: processes are assumed to be small until a reference at a higher virtual address is actually observed. At that point, the process is graduated to large procss status. While all of this is transparent to the application, authors of performance-critical applications may find it helpful to know where the magic addressing threshold is.

The 32 page restriction is motivated by internal implementation convenience in the kernel. Future kernels may provide greater flexibility in this regard.

In order to play nicely with the small space optimization, the ``small process'' runtime library typically sets up the process stack to begin at 128k and grow downward.

3.2 Memory Allocation

Most programs find it convenient to assume that memory allocation will be handled transparently by a memory keeper. Typically, these programs are fabricated as ``virtual copies'' (copied via copy on write) of some original program image. The first time an existing page is modified, a private copy of that page is made for the application. Similarly, when a previously undefined address is first read, a new, empty (zero) page is allocated ``on demand'' for that location.

This model is convenient, because heap and stack growth are transparently handled, but this convenience comes at a cost. A per-address-space memory keeper must exist, and this keeper must be instantiated before the program runs.

To minimize overhead, some programs may find it more appropriate to manage their own memory. This is actually important enough that some support for it exists in the core runtime libraries.

Self-managed memory raises three concerns:

• If the stack overflows, there is no keeper to catch the overflow. A small program author must therefore know in advance how deep the stack may become.

• When the heap grows, the application must arrange to allocate the necessary pages and place them in the address space at the appropriate locations.

• Because there is no memory keeper for a self-managed process, the teardown protocol for the process address space must be different from the teardown protocol for kept spaces.

The runtime library deals with the memory management issues via three variables that can be overridden by the application as needed. The three variables and their defaults in each runtime model are shown below:

Each of these symbols is defined as a weak symbol by the standard runtime libraries. The normal case values are defined in crtn.o. Applications built with -lsmall receive overridden versions of these variables corresponding to the values shown for the small runtime above.

During it's lifetime, an application may well switch from one mode to another. For example, an initially small-model application may allocate a kept heap area, set up the heap there, and change the value of __rt_kept_heap accordingly. Mechanisms for managing heaps in this way are included in the -lselfheap library.

Programmers are advised that the heap support routines compiled into the -lsmall library presume that the application has been created using mkimage as a small process. The functions provided in the conventional runtime library are more general.