[E-Lang] E modules

[Mark Seaborn]

A few weeks ago, Darius Bacon e-mailed me about module systems for E.
I replied, and Mark Miller said he'd like to see the discussion on the
e-lang list.  So here's the three e-mails (I hope you two don't mind
me forwarding your e-mails here):


Darius Bacon <darius@accesscom.com> wrote:

> Hi there, we haven't met but I was intending to write you about module
> systems sometime.  Mark Miller asked me a while back if I'd be
> interested in drafting a module-system proposal for E, and I am, but
> have a compiler as a higher priority.  When the matter came up again I
> intended to start out by seeing if we could simplify PLT Scheme units
> using promises for mutual recursion, but that's as far as I've thought
> it out, and I remembered from your web page that you'd been thinking
> about this kind of stuff.  So it's convenient that you've discovered E
> too!

I replied:

> Okay, here's an attempt to go through some of the issues in a
> reasonable order:
> 
> The problem with PLT Units is that it doesn't make unit invocations
> first class.  You only get to link unit invocations together using the
> language of `compound-unit', which isn't very expressive.  You can
> make unit invocations first class, which simplifies things a lot,
> though you start to lose guarantees about how linking works, what it
> means, etc., particularly in a very typeless language like E (it's
> more typeless than Scheme in the sense that bits of data are just
> objects in E).  I'm not sure how much of a problem that would be.
> 
> Unit invocations can be reified in the form of promises, as you
> suggest.  Actually, I've only just started thinking of the
> consequences of that this afternoon!  In my experimentation with
> extending Units, I have always used logic variables so far.  But
> separating the authority involved into promise and resolver
> capabilities is better.  Promises also have a notion of direction --
> resolving a promise to itself can be caught as an error, whereas
> unifying a logic variables with itself cannot.  Logic variables have
> to be done in a separate layer in languages without them, although
> that does mean you can put extra checking into that layer,
> eg. constrain a logic variable in advance to conform to a signature.
> Perhaps constraints can be extended to promises/resolvers.
> 
> In PLT Units, units can convey authority, because they can capture
> bindings lexically when they are created.  This should be avoided,
> depending though on what you mean by `unit'.  We need some good terms
> for the separate stages of the linking process:
> 
>  * A unit starts off as a piece of data, conveying no authority.
> 
>  * This piece of data gets evaluated (either using the E
>    evaluator/compiler, or it could specify another program to compile
>    it into E by giving a hash of this program -- more on that later).
>    If the piece of data wasn't a lambda expression, the value returned
>    might convey authority, since the code might create a mutable
>    object.
> 
>  * The resulting function gets called with some unit invocations as
>    arguments, and returns a unit invocation.  (What a unit invocation
>    is is defined by that sentence!)  If the function might convey
>    authority, it should be discarded so that unit invocations do not
>    unintentionally have a means to communicate with each other.
> 
> The first two steps correspond to how e-makers work, as I understand
> it.
> 
> A unit invocation could be treated as a dictionary mapping names to
> values.  In the above scheme there's no way to tell what signatures
> the inputs and outputs of a unit should obey, so a unit could be
> extended to be the tuple (in-signatures, out-signature, body-expr), or
> even better, an expression of the form:
> 
>   def _(a1 : in_signature_1, ...) : out_signature { ... }
> 
> so that the user of the unit can determine the unit's signature
> statically (from the unit's source).  This would expand to give
> runtime checks, as in E (but some obvious mismatches can be spotted
> statically).
> 
> Signatures would just be sets of names (names could be associated with
> E guards; since guards could include signatures, this would allow
> nested signatures as in PLT Scheme's signed units, although I don't
> think they're very useful).  PLT Units doesn't handle the issue of
> naming signatures very well; it ends up with a universal namespace of
> signatures at the Scheme macro-expand level.  A pet name system for
> signatures could be used, although I don't know how it would fit in
> with editors.  At any rate, units should have their signatures inlined
> before using or distributing.
> 
> Some extra syntax might be needed for accessing dictionaries
> conveniently (like ML's dot-notation).  (Units seems to go out of its
> way to avoid that.  It might be because binding different functions as
> different names rather than naming them through a dictionary is more
> like C and can be done more efficiently.  Or it might be because the
> designers of PLT Units didn't want to extend Scheme's reader.)
> 
> MarkM suggested on the e-lang list using hashes to name chunks of data
> like units (this could be extended to include the size of the data as
> well as its hash so we have an idea in advance about the cost of
> retrieving it).  Suppose a compound-unit form (or its equivalent)
> refers to some units by their hashes.  It would be awkward (and
> perhaps risky) to give the compound-unit form a capability with which
> to look up hashes.  It would be better to include the hashes directly
> in the compound-unit expression using some special syntax, and later
> expand the compound-unit expression to include the source of the units
> it links.
> 
> Putting a hash into a compound-unit form is effectively the same as
> inlining the source (in naming terms, but not resource terms), and
> this inlining can be done in a similar way to inlining of signatures
> (ie. exact details not worked out, just the idea of editing code in a
> form friendly to editing, and linking code in a form where naming is
> unambiguous and does not involve any context).
> 
> Also I have some ideas about how units can help typed code coexist
> with untyped code (units would be better than ML's module system for
> this), but I'll save them for later.
> 
> 
> > This is just a heads-up in case you and MarkM have not discussed it
> > already.
> 
> No, I'm quite new to discussing E. :-)
> 
> Do you want to move this discussion to the e-lang list?  Also, I'm
> going away for the next week, but I can elaborate on some of the
> things above that are probably too vague when I get back.


Mark Miller replied:

> I would very much like to see this discussion on the e-lang list.
> People there that would have strong opinions about these issues
> would include at least Dean, Jonathan Rees, and Ralph Hartley.  In
> addition, the resolved name -> hash issue brings up possible issues
> regarding interaction with configuration management systems, for
> which at least Jonathan Shapiro will have strong opinions.  Posting
> to the list is the cheap way to figure out who we should include.

-- 
         Mark Seaborn
   - mseaborn@bigfoot.com - http://www.srcf.ucam.org/~mrs35/ -

                  How to write good:
          ``7. It is wrong to ever split an infinitive.''