[e-lang] Designing the E module system (was: A thought on module systems)

[Thomas Leonard]

On Fri, 2010-09-03 at 06:52 -0400, Kevin Reid wrote:
> On Aug 29, 2010, at 4:01, Thomas Leonard wrote:
> 
> > On 29 August 2010 03:31, Kevin Reid <kpreid at switchb.org> wrote:
> >> I think this is an original thought I had about the problem of  
> >> modules/
> >> libraries:
> >>
> >>   A module (library) is a stateless unum.
> >>
> >> Therefore:
> >>
> >>   Module loading is obtaining a presence of the unum.
> >>
> >>   Module naming is long-term remote object naming.
> >>
> >>   Library versioning is an upgrade/protocol-compatibility problem.
> >
> > By the way, I implemented such a module loading scheme for the ebox  
> > demo:
> >
> >  http://0install.net/ebox.html
> 
> Just to get *some* of my thoughts on the matter in writing so we can  
> maybe make some progress, I'm going to comment on what I see :
> 
> > For example, the "ebox-edit" example application declares a dependency
> > on the "ebox-help" library like this:
> >
> >    <requires interface="http://0install.net/tests/ebox-help.xml">
> >      <environment insert="" name="help"/>
> >    </requires>
> 
> This is good, naming libraries by URLs, and giving the client control  
> over how the library is made available. What does the insert=  
> parameter mean?

It's the directory within the package which contains the root of the E
module. It's confusing and badly named in the context of E modules (and
probably always ""), but in other contexts you might use it like this:

   <requires interface="http://repo.roscidus.com/java/openjdk-6-jre">
     <environment insert="bin" name="PATH"/>
   </requires>

to get the Java package's "bin" directory inserted into your $PATH. I
should probably have created a new element, but it was easier to
re-purpose the existing <environment> ;-)

> We should not use XML, because XML is designed for document markup,  
> and we have a better choice at hand; namely TermL. (Though there is no  
> formalized TermL feature serving the role of XML namespaces, yet.)

One nice thing about XML is that if you visit the URL in a web-browser
then an XSLT stylesheet formats it as HTML for you.

> > This causes a loader for the library to be added to ebox-edit's
> > environment (Scope), so that it can import things using e.g.
> >
> >  def makeHelp := <help:makeHelp>
> 
> Does <help> necessarily expose every file in the module as an  
> importable item? It should be possible for a module to have private  
> components, so that its public interface is well-defined.

You could modify it to pass <help>.getPublicAPI() instead of just <help>
to users of the package. But without DeepFrozen that has issues of its
own.

Another option is to split out the API into a separate directory and
depend on it using:

  <environment insert="api" name="help"/>

Then, help's public API could depend in turn on code in the "impl"
directory. Seems a bit heavy-weight though.

> What about the module's access to itself? In particular, a module  
> should have a reference to its own file tree (as in the <resource>  
> loader), or a customizable file-to-object mapping thereof, so that it  
> can efficiently store binary data (e.g. images for a GUI library).

All modules get access to their own loader as <this>. You can use
<this>.getRoot() to get a read-only file object for the root directory
of the module.

By the way, the launcher scans all modules at start-up to check that
they don't contain symlinks. Would be nice if E's file objects handled
that more safely, though I can't see how to do it.

> > The "interface" URI is both the name of the library and a hint about
> > where to get information about it.
> 
> Good.
> 
> > Incompatible changes use a
> > different URI,
> 
> Good.
> 
> > while backwards-compatible changes just use a different
> > version number. Versions can be restricted like this:
> >
> >    <requires interface="http://0install.net/tests/ebox-help.xml">
> >      <environment insert="" name="help"/>
> >      <version not-before='1.0' before='2.0'/>
> >    </requires>
> 
> What is the grammar and ordering of version numbers?

Version := DottedList ("-" Modifier? DottedList?)*
DottedList := (Integer ("." Integer)*)
Modifier := "pre" | "rc" | "post"

See: http://0install.net/interface-spec.html#versions

> > Downloads get shared automatically, so that if two programs happen to
> > use the same version of the same library, then the library only gets
> > downloaded and stored on disk once.
> 
> It is also important that if there is a diamond dependency
> 
>    A requires B --> B requires D
>    A requires C --> C requires D
> 
> then only one copy of D is loaded into memory, so that B and C have a  
> shared vocabulary of objects.

If two modules have dependencies with the same interface URI then it
selects a single version for both.

Satisfying all the version constraints in this case was challenging, but
it solves the problem by expressing all the version requirements as a
huge boolean expression and using a SAT solver to find the solution.
See: http://roscidus.com/desktop/node/954


-- 
Dr Thomas Leonard
IT Innovation Centre
2 Venture Road
Southampton
Hampshire SO16 7NP

Tel: +44 0 23 8076 0834
Fax: +44 0 23 8076 0833
mailto:tal at it-innovation.soton.ac.uk
http://www.it-innovation.soton.ac.uk