Jon:
How about the scheme used for the UNIX file system's inodes? It deals with exceptional cases (large files) with indirection to elements of the backing store, which are brought into memory as needed. This relieves the memory pressure while ensuring performance in the common case. In your context, there might be a descriptor which refers to block numbers on the backing store. The file system's buffer cache has some nice ideas, some of which might be absorbed.
-Jonathan
At 3:55 PM 11/21/94 -0500, Jonathan Shapiro wrote:
>I am trying to write up an architectural description of the new
>system. In it, I have arrived at a difficult design problem and would
>welcome suggestions.
>
>I also need a name for the new system. Parly because it's such a bad
>pun, I'll call it NewSys here.
>
>Like Mach, NewSys implements a notion of a memory object. A memory
>object has a length in bytes, and can be mapped at a particular offset
>in a process address space. More specifically, the mapping has the
>form:
>
> (MemObCap, MemObStartPage, VPageAddr, PageCnt, Permissions)
>
>where
>
> MemObCap -- capability for the memory object
> MemObStartPage -- offset, in pages, to the location in the
> memory object at which the mapping should begin.
> VPageAddr -- virtual page address at which the mapping
> should appear
> PageCnt -- number of pages to be mapped
> Permissions -- permissions associated with the mapping.
> As with KeyKOS, permissions are
> subtractive; if the memory object is read
> only, you are free to map it read write,
> but it won't get you anywhere to do so.
> [ Actually, should this generate an error?
> If you are handed an RW capability to the
> object at a later time you will need to
> remap anyway... ]
>
>The problem of the short last page in the object has well defined, if
>unfortunate, semantics.
>
>Behind the scenes, what happens is that you touch a page that the
>kernel doesn't have mapped into your mapping tables, it requests the
>data from the memory object's manager. In Mach, this information is
>cached, so that an object mapped by more than one process will in
>general produce a manager data request on a particular object page
>once.
>
>In NewSys, the memory object description will be stored in an in-core
>"object frame" (somewhat analogous to a KeyKOS node -- only object
>frames can contain capabilities, and they cannot be mapped into user
>space). Object frames cannot be paged out by the kernel, because the
>kernel does not implement persistence. [Eventually, I will probably
>introduce a mechanism that allows the kernel to ask an object's
>manager to remove an object from memory, and part of that protocol
>will be for the manager to supply a new key that should replace all
>outstanding keys to the in-memory object.]
>
>One would like to provide a mechanism whereby the memory object
>manager can provide an optional storage map to the kernel describing
>where the disk storage for the object can be found. This storage map
>would allow the kernel to service memory object I/O directly, and to
>provide ageing and pageout services for memory object data. The
>storage map needs to contain disk page keys, and must therefore be
>built out of object frames. The problem is that these object frames
>are not pageable in the NewSys system, so one would like this
>descriptor to be discardable and have it be regenerated when necessary
>by the manager. This is further complicated by the need to provide
>remotely backed memory objects.
>
>I'm tempted to adopt something very like the KeyKOS segment
>architecture for the storage map, but I'm concerned about memory
>pressure.
>
>Ideas?