Recent attempts to port drivers to EROS have motivated the removal of drivers from the EROS kernel. This note describes the necessary changes to the EROS kernel in order to achieve this.

Introduction

User mode drivers are potentially more robust than their supervisor mode counterparts. Supporting them is a bit tricky, however:

1. Initial drivers must be loaded before the kernel begins execution, and we must ensure that they are not removed from memory by the ageing logic.

2. User-mode drivers must have sufficient privilege to issue commands to hardware-level I/O registers.

3. DMA from user mode is challenging, because the DMA device generally wants physical addresses. This in turn tends to mean that there must be an arrangement by which the driver obtains some memory whose physical address(es) it knows.

4. Because of faulty hardware, user-mode drivers must sometimes be allowed to disable interrupts and prevent themselves from being preempted.

5. Interrupts must be demultiplexed to the appropriate process.

6. Because of the checkpoint mechanism, the kernel must sometimes be able to perform an ``upcall'' to a user mode driver. Also, the kernel must have an exclusive relationship with the disk drivers. These is the only issue that is particular to EROS.

I will address each of these issues in turn.

Straightforward Issues

The first five of these issues are straightforward in principle.

Driver Load

We will revise the bootstrap process in such a way that the bootstrap loader, in addition to loading the kernel, also loads an initial object range that contains the drivers. This range will be a ``well known'' object range.

Objects in this object range are exempt from checkpoint, ageing, and such, and the drivers in them therefore will not be accidentally removed. These objects can be explicitly released from memory by the driver manager once all drivers have been configured. After this their pages are subject to removal by ageing.

For the sake of simplicity, we will assume for the moment that no drivers will be released from memory in this way. For embedded and set-top devices, we probably want to build system images that contain only necessary drivers in any case.

I/O Registers

We assume that these programs will be privileged, in that they are permitted to directly manipulate I/O registers. This is accomplished either by initial capability calls that validate portions of the I/O register space or by simply gifting these processes with extraordinarily privileges on the grounds that they live within this distinguished object range and are therefore part of the operating system.

User-Mode DMA

Drivers are expected to program DMA devices. We will implement a special kernel capability that allows a driver to say to the kernel: ``Here is a range of contiguous virtual pages. Arrange for them to be physically contiguous within the following physical address range and tell me what the resulting physical base address is. The resulting information is sufficient to allow the driver to directly program physical DMA devices.

It is anticipated that some drivers will contrive to map these buffers in such a way that the driver virtual address is the same as the resulting physical address in order to avoid the need to translate between driver virtual and kernel physical addresses.

Faulty Hardware

Unfortunately, there is faulty hardware in the world. Some of this hardware requires that interrupts be disabled for reasons other than temporary mutual exclusions. The CMD640 PCI/IDE bridge for example, has a bug in which an interrupt occurring during data transfer can result in corrupted data being written to the disk.

On such systems, we will provide means for driver processes to disable interrupts. This of course renders the entire system vulnerable, but it is a necessary response to the broken hardware. We can at least limit the number of drivers that require this authority.

Interrupt Demultiplexing

Drivers will register for the interrupts they wish to receive. The kernel will demultiplex interrupts and then wake up the appropriate process to handle the result.

Checkpoint and the Object Cache

For a long time it stumped me how to move disk drivers out of the kernel. Charlie Landau asserted in December that he thought he saw how it might be done, but I could not convince myself that I was happy about the only solutions I could envision. It now appears that there is actually a very natural ``cut line'' in the design for this, and a perfectly reasonable approach to the implementation.

The idea is to entirely remove all responsibility for disk I/O from the kernel. The kernel retains responsibility for checkpoint directory management, but at the very bottom level it performs upcalls to accomplish the following operations:

• ReadObFrame(oid) requests that the storage system should read the object frame at OID and install the resulting page into the object cache via a kernel object cache capability call.

• ReadLogFrame(lid) similarly for log frames.

• WriteObFrame(oid) requests that the storage system should write the bytes passed via the associated IPC at the appropriate disk location(s) for this oid.

• WriteLogFrame(lid) similarly for log frames.

One or more processes sits outside of the kernel in the following loop:

for(;;) {
  call kernel to get next upcall
  process upcall
  acknowledge completion and wait for next upcall
}

Why Use Upcalls?

In other parts of the EROS kernel, a different approach to upcalls is used. When a process takes a fault, for example, the kernel synthesizes a keeper invocation by making it appear that the faulting process has directly invoked the keeper. Unfortunately, this approach cannot be used for driver upcalls because the drivers are not persistent.

When a CALL occurs or is synthesized, the recipient gets a resume key to the caller. The problem here is that drivers are not persistent, so this resume capability can be lost. Also, if driver capabilities are simply start capabilities, there is no simple way to rescind them on restart. The problem is inherent in any call that crosses the boundary from persistent to non-persistent code.

The solution is to arrange a rendevous mechanism within the kernel for rendevous. The driver calls into the kernel and is queued waiting for a response (i.e. a return from the kernel. When the client invokes the driver key, it is in turn queued waiting for a response and the inbound message is passed to the driver. Later, when the driver calls back into the kernel, it's inbound message is passed to the client as a response.