[e-lang] Kevin's thoughts on modules

[Thomas Leonard]

On 11/04/11 03:08, Kevin Reid wrote:
> My rough ideas for a module system include these goals/characteristics
> which are notably different from emakers:

Here are a few comments on how this relates to what I've already 
implemented (and is in 0.9.3). The existing system is implemented in E, 
so it should also work on other systems, though I haven't tried it. 
Documentation is here:

   http://wiki.erights.org/wiki/ELoader

There are three ways of using it:

- On its own (just the code implemented in E 0.9.3)
- With 0install
- With ebox

ebox uses 0install to get all the dependences and wire them together, 
and then runs the program in a restricted environment (with a powerbox 
to request extra access).

Using it with 0install (but without ebox), the program itself is run 
unconfined but its libraries get no special privileges. This is 
necessarily if you need more than the sandbox provides (e.g. access to 
an untamed Java library).

> * No global namespace as in Java classpaths. (In Java, only convention
> prevents a .jar from stepping on any name it wants to; this should be
> *inexpressible* in a capability system.) Instead, once a 'module' has
> been identified, it has its own namespace which is free to be entirely
> flat (or not).

ELoaders already work like this.

> * It should be possible to identify a module in a highly future-proof
> way, and have it automatically downloaded if needed. In particular, it
> should be easy to say "def myModule :=<load the code which has the
> SHA-whatever hash code 324587.... plus the following locator hints>".

Already supported, except the SHA hashes don't go in the code, but in
a separate package metadata file (XML). That also means you can depend 
on a range of possible versions, or different versions for different 
environments (e.g. an implementation localised for the user's preferred 
language).

Hard-coding exact versions would make it hard to fix bugs in libraries, 
because all code using them would need to be edited. However, you can 
put version restrictions in the XML if needed.

You can also, outside of the program, snapshot the exact library 
versions being used and run again later with the same versions.

> Goals:

>     - that it be rare that one wishes to cut-and-paste code (without
>       modification) rather than loading it as a module.
>
>     - that there is little barrier to using libraries which are not
> part of
>       standard E distributions.
>
> It would also be nice to, while being all nice and integrated about
> module downloading, avoid inventing the equivalent of another package
> installation system (CPAN, ruby gems, py-
> whateverthey'redoingthesedays, quicklisp, etc.)

Already working (using 0install). Works on Windows too now, not just 
POSIX systems.

> * A module should be able to have private shared objects while its
> implementation is broken across multiple files. (In the emaker system,
> everything is either public or local to one .emaker file.)

It doesn't do this.

> * A module should be able to efficiently contain objects which are
> unserialized (in the most general sense) data (e.g. images) without
> exposing the distinction between these and code to clients.

Already done.

> There are probably other requirements I haven't remembered of just now.
>
>
> Open questions is when modules should be instantiated (is there one of
> a given module per vat, or not?); if they can be multiply
> instantiated, whether that's 'deep' or 'shallow' wrt dependencies
> (where do you stop?); and how to support (if at all) modules or graphs
> of modules being granted references to privileged/application-specific
> objects. (It would be nice to say that a module is always
> unprivileged, but that results (I think) in certain types of
> applications requiring massive application of the Author pattern,
> which is undesirable in my opinion.)
>
>
> The last time I was thinking about this, I actually wrote some code;
> see the darcs repository at<http://switchb.org/darcs/e-modules/>.
> This is not a complete system and probably has some design flaws in
> its module format; but it works enough that I successfully loaded Den
> in it.
>
> Whatever actually gets done should exist directly in E
> implementations, not be an outside library, of course.


-- 
Dr Thomas Leonard
Gamma House, Enterprise Road,
Southampton SO16 7NS, UK


tel: +44 23 8059 8866

mailto:tal at it-innovation.soton.ac.uk
http://www.it-innovation.soton.ac.uk/