[E-Lang] Syntax change: reducing side-effects

[Mark S. Miller]

Terminology: Until we decide whether or not to incorporate "var" into E, in 
order to keep things straight, let's call Dean's proposed variant of E 
"Evar".  I have a working (unreleased) Evar, which I used in the examples 
below.


At 04:47 PM Sunday 2/11/01, hal@finney.org wrote:
>With the proposed convention I gather I'd have to add the "var"
>declaration to all my function names.  Then when I had them all working
>the way I wanted I'd have to go through and take all the "var"s out.
>
>I've always liked interactive development systems that let you work like
>this, going back to APL.  You define a function and then try calling
>it from the command line, and when it works you define more complex
>functions that use it.  Does anyone else use E this way?

Hal, I'm really glad you raised this, because in all the discussions so far 
between Dean, MarcS and I, none of us ever noticed the issue.  E is indeed 
designed to support this kind of interactive shell/command-line usage, 
including the redefinition of previously defined functions/objects.  In 
fact, I'll go ahead and call this a "requirement" since I'd be really 
unhappy if we lost this functionality.  When I first read your message I was 
worried.

Then I realized all is well.  Your example isn't assigning to these 
variables, it's redefining them.  In general E does not allow redefinition 
of the same variable name in the same scope.  The top level interactive 
scope is the one exception to this 
http://www.erights.org/javadoc/org/erights/e/elang/scope/MutableScope.html
, in order to support exactly the style of interactive development / rapid 
prototyping you're using.  Since the "var" change only affects assignment 
and not definition, this special interaction between redefinition and the 
top level interactive scope should be unaffected.  Here's an Elmer on Evar 
session demonstrating these points.  I removed the stack backtrace from the 
problem reports below.

For background, see http://www.erights.org/javadoc/org/erights/e/elib/slot/ValueGuard.html and http://www.erights.org/javadoc/org/erights/e/elib/slot/SlotGuard.html .


    ? {
    >     def a := 3
    >     def b := 4
    >     def a := 5
    > }
    # problem: <AlreadyDefinedException: conflicting definition of a>
 
This demonstrates that a non-top-level scope does not allow a variable to be 
redefined.


    ? def x :integer := 33
    # value: 33

This "x :integer" is equivalent to the E "x :final(integer)" 
or the Evar "var x :final(integer)".

    
    ? x := 44
    # problem: <ClassCastException: org.erights.e.elib.slot.FinalSlot>

We should have a more informative error message for assignment to a final 
variable.  Actually, E and Evar specify that a program with such an 
assignment must be statically rejected, and not allowed to begin execution. 
Postponing the error check till runtime is an old known bug.  In E, it was 
never high priority to fix this bug, because most variables were implicitly 
mutable anyway.  In Evar, this static check should be much more useful.

    
    ? def func() { println(x) }
    # value: <func>

    ? func()
    33

This function definition uses "x" freely, thereby capturing "x" as an 
instance variable.


    ? var x :char := 'a'
    # value: a

In answer to Hal's worry, this redefinition succeeded even though the above 
assignment failed.  By being defined as final, x could not be reassigned, 
no matter what scope it's defined in.  By being defined in the top level 
interactive scope, x can be redefined, independent of what it's defined as.


      Yet Another 2x2 Matrix: (Scope, Slot) x (Immutable, Mutable)


To understand the semantics of this, let's make explicit the two levels 
needed to map a variable name to the value of that variable.  For these 
purposes, we will refer to the above expression as doing two things: 
*defining* the variable to be represented by a Slot, and *initializing* the 
Slot with an initial value.


* Evaluation happens in a scope.  A scope is a mapping from names to Slots.  
A variable definition creates a new name => Slot mapping. A variable name 
expression is evaluated in a given scope by first looking up the Slot bound 
to that name in that scope.

Scopes are normally immutable, so the Slot for a given name in a given 
scope cannot be changed -- normal scopes do not allow redefinition.  The 
exception is MutableScope, expected to be used only in the top level 
interactive environment.


* A Slot responds at least to getValue().  MutableSlots additionally respond 
to setValue(newValue).  We refer to the result returned by calling 
getValue() on a given Slot as "the Slot's value".  A Slot's initial value is 
expected to be a coercion of the specimen provided to the SlotGuard that 
made the Slot (eg, the value of the right side of a "define"), but that's up 
to the SlotGuard and the Slot it makes to decide.

To evaluate a variable used as an expression (rValue), the implementation 
calls getValue() on the Slot obtained in the first step.  Whatever it 
returns is the value of the expression.  To evaluate an assignment to the 
variable (lValue), the implementation calls setValue(newValue) on the Slot, 
where newValue is the value of the expression to the right of ":=".  Of 
course, following an assignment, the Slot's value is expected to be a 
coercion of newValue, but that's up to the Slot to decide.

final Slots are immutable, so the value of a final Slot won't change.  The 
final Slot does not allow assignment.  In Evar, Slots bound to variables are 
normally final.  The exceptions include only user-specified SlotGuards, for 
which the "var" syntax is required.


To repeat for context, and to demonstrate that it's harmless in Evar...

    ? var x :char := 'a'
    # value: a

In Evar, "var x :char := 'a' " is shorthand for "define var x :char := 'a' ".

In the expanded form, the "var x :char" is equivalent to the E "x :char". 
"char" obeys the normal type-like guard convention: It implements both 
ValueGuard and SlotGuard protocol.  As a ValueGuard, it only accepts values 
it can coerce to the type it represents.  As a SlotGuard, it is equivalent 
to "settable(char)", which makes a SlotGuard that makes a settable 
(assignable) slot that coerces assigned values using "char" as its ValueGuard.

Therefore, *given the behavior of "char"*, "var x :char" is equivalent 
to "var x :settable(char)".


    ? x := 'b'
    # value: b

    ? x := 44
    # problem: <ClassCastException: java.math.BigInteger doesn't coerce to a 
                       java.lang.Character>

These demonstrate that x's nature seems completely based on the second 
definition and not the first.  It is now settable (assignable), whereas 
previously it was final.  It is now constrained to hold only characters, not 
integers, whereas previously the opposite was true.  This complete erasure 
of the old definition with the new is needed for this kind of interactive 
prototyping.  However, it raises a potentially nasty question.  


                                            The Strange Case


When "func" was defined, we say it captured (as an instance 
variable) the variable "x", but what exactly did it capture.  A plausible 
hypothesis would be that, on creating the object bound to "func", the Slot 
for "x" was looked up and that "func" contained it's own private mapping 
from "x" to this Slot.  

Unfortunately, this story would make Hal's style of prototyping very 
tedious.  On redefining anything, Hal would need to redefine everything in 
its scope.  Fortunately, this isn't what E does.
    
    ? func()
    b

Here we see that "func" has looked up the Slot bound to "x" by the most 
recent assignment, rather than the most recent prior to its own definition.  
To get this right, "func" actually captures a reference to the outer scope 
itself as an instance variable, and actually looks up the names at runtime 
(by sending messages to this scope object).  There are of course many 
opportunities for transparent compiler optimization here, but when the outer 
scope is mutable, not as many opportunities as one might expect.  OTOH, the 
other scope should only be mutable in an interactive context, which is 
hardly the most demanding for speed.


                            Please indulge me while I brag


Needless to say, it was not easy to work out the above Scope vs Slot level 
separation, the behavior of the top-level interactive scope, satisfy the 
rapid prototyping requirements, and satisfy the compiler requirements for 
static analyzability, all within a simple orthogonal semantic story that 
most programmer can safely avoid ever learning.  Whew, it's even hard to 
say.  However, this separation is about two years behind me now, and has 
worked so perfectly during that time, that I simply forgot about it when 
discussing "var" with Dean and MarcS and when building the Evar used above.  
Until Hal's message reminded me.  The fact that this Evar works anyway gives 
me some confidence I got this separation right.

It took Dean, MarcS, and I weeks of occasional argument to arrive at an Evar 
proposal worth taking seriously.  But once it settled, it took me less than 
an hour to implement.


        Cheers,
        --MarkM