[e-lang] Alternative E loader

[Kevin Reid]

On Feb 11, 2010, at 11:50, Thomas Leonard wrote:

> Hi all,
>
> The E documentation (e.g. the Walnut guide) suggests that .emaker  
> files
> should be placed somewhere in classpath and then imported using
> <import:mypackage.myMaker>.
>
> If I want to depend on someone else's library then presumably I'm
> supposed to add their jar or directory to my classpath. However:
>
> - The classpath is a global (process-wide) namespace, so their emakers
> may override or conflict with existing ones.
>
> - Their jar may also introduce other files that affect the whole  
> process (e.g. .safej files, giving their emakers access to unsafe  
> Java code).
>
> - I have to mess around with extra -cpa options in the launcher  
> scripts
> (which always has to be duplicated, to support Linux and Windows).
>
> - I can't add extra modules at runtime.
>
> - I can't depend on two libraries that use the same class names (e.g.
> two different versions of the same library).


Yes, these are all problems with the current system. Fixing it has  
been on my mind for a while, under the name "module system".

The old draft code is at
   http://switchb.org/darcs/e-modules/

I really need to write further about this, and consider your  
contributions, but I haven't found the time to do a thorough review  
since you wrote this first message; I'm just informing you of the  
current situation and prior work.

On Feb 19, 2010, at 10:45, Thomas Leonard wrote:

> If accepted, it could also be used by rune to load the main program.
> This would allow all E programs to import emaker files from the same
> directory without any extra configuration (e.g. rune -cpa ...).

Per capability design principles, programs should not get automatic  
read authority to what happens to be in the same directory. However, a  
"run the program which consists of this module-directory's main entry  
point" or other invocation mechanism which explicitly specifies *that  
the directory access should exist* would be reasonable.

> In terms of speed, it is dramatically faster than <import>. This test
> case imports "testMaker.emaker" 100 times using both methods:

This is surprising. If true in general, then fixing whatever is slow  
about <import> would probably significantly speed up a lot of E code.

Also, I note that your system still re-evaluates on every get/1;  
eliminating this (though it raises shared-state issues until such time  
as the results are audited DeepFrozen) ought to also be a speedup for  
the situation where many other files use the same library emaker.

> - The getRoot method allows emakers to discover their own location. If
> this is a problem, we could replace it with "getTextWriter" and
> "getOutputStream" instead.

I have thought for a while that there ought to be an operation on file  
objects to hide their pathnames, making them "virtual filesystem roots".

> - Rather than creating a whole new scope for each file, I just extend
> safeScope. This avoids having to keep reloading things. I'm assuming
> that safeScope's slots can be safely shared between files?

This is correct.

A number of additional points:

(1) The name "scope" is deprecated in favor of "env", as a matter of  
terminological correctness; the corresponding code changes have not  
been made, but new code should use the proper names within itself. (2)  
Your explicit env manipulation can be simplified by using the  
evaluation of code (untested example):

def [traceln__Resolver, fileScope] := e`
   def traceln
   traceln__Resolver
`.evalToPair(safeScope.withPrefix(...))
bind traceln := makeTraceln(fileFQName)
return fileScope.nestOuter()

However, this technique has the disadvantage of putting a spurious  
traceln__Resolver variable in the env.

(3) You should not bind "safeScope" to the file env; the name  
"safeScope" (should be safeEnv) denotes the *particular* env which has  
no interesting authority, whereas yours does.

(4) When setting up an env, always do a nestOuter() last; this  
operation makes all of the variables considered to be in an enclosing  
scope rather than the current one, so that they are permitted to be  
rebound. (This is the subsystem which prohibits { def x := 1; def x :=  
2 } but permits { def x := 1; { def x := 2 } }.)

(5) Code which manipulates envs using state maps should be avoided, as  
this discards the “binding” guard information which will be required  
by guard-based auditing. However, there will arguably be larger  
changes here as E-on-CL and E-on-Java are reconciled in the area of  
env manipulation.

-- 
Kevin Reid                                  <http://switchb.org/kpreid/>