[e-lang] Making sense of the "Non-local Exits vs Defensive Consistency" thread

David Hopwood david.nospam.hopwood at blueyonder.co.uk
Tue Jan 30 00:36:19 CST 2007

Dean Tribble wrote:
>  small note: changing the subjects broke subject threading in gmail :<

The following summary might help:

I had posted a response to a thread on Lambda the Ultimate, talking about
a proposal for transactions in E:

(That post incorrectly says that the proposal is for "speculative"
transactions. In fact they are not speculative: transaction rollback would
only occur on an exception. Speculation may be appropriate for database usage
patterns, but I think that the assumption that concurrent transactions usually
do not interfere is less likely to be applicable to programs in general-purpose
programming languages.)

Originally I promised to post the proposal that week, but work and then Christmas
got in the way. The proposal is much improved for having waited; I understand
the implementation issues and how to analyse programs in this model better now.

MarkM politely reminded me about this:
> I remain curious about your thoughts on speculative concurrency for E.

I replied with the summary article that starts with the heading
"Transactional Event Loop Concurrency". The other articles in this series
haven't been posted yet.

There was then then a physical meeting at HP (I'd have loved to have been
there, but never mind.) MarkM, MarcS, Tyler Close, David Wagner, Alan Karp, and
Adrian Mettler were cc:d to the resulting e-mail thread; I'm not sure whether
they were all at the meeting.

Tyler wrote:
> At the Friday morning cap meeting today we spent a lot of time thinking
> about programming for defensive consistency in Joe-E, in the presence of
> VirtualMachineError. The conclusion was that it's not feasible with the
> current Joe-E rules, and some form of amendment is needed.

with an example in which an OutOfMemoryError (a subclass of VirtualMachineError)
occurs while an object is in an inconsistent state.

> We went through a bunch of proposals for fixing this before deciding
> that the best thing to do is forbid catching of a VirtualMachineError,
> and forbid finally clauses (since they can result in a
> VirtualMachineError being silently dropped). We found coding idioms to
> replace the common uses of finally clauses, so that although extreme,
> this loss is survivable.
> The finally clause could be added back to Joe-E if MarkM's JSR
[Java Specification Request]
> suggestion for adding a Keeper API to Java goes through.

I'd still like to hear more details on this JSR proposal.

David Wagner replied:
> Forbidding 'finally' is pretty devastating and seems out of proportion.

There was general concensus on this.

> The solution MarkM had recommended earlier was a VM modification to cause
> the VM to terminate immediately rather than throwing a VirtualMachineError.
> I'm curious to hear what made you reconsider this stance.
> If you don't like this, there is only one other alternative I've seen that
> I believe to be viable.  The Joe-E verifier would check the following rules:
> 1) Any 'catch' clause whose declared type is VirtualMachineError, or
> a subtype of VirtualMachineError, must have exactly the following body:
>   catch (VirtualMachineError e) {
>       java.lang.Runtime.getRuntime().halt(1);
>   }
[snip technical points about not using System.exit and about the security manager]

> 2) Any 'catch' clause whose declared type is Error or Throwable
> (the supertypes of VirtualMachineError, other than Object) must be
> preceded by a 'catch' clause whose declared type is VirtualMachineError
> (and thus is of the above form).
> 3) Every 'finally' clause must be preceded by a 'catch' clause whose
> declared type is VirtualMachineError (and thus is of the above form).

[a necessary additional rule was pointed out by MarkM:]
> * Joe-E code cannot declare and direct subclasses of Throwable, i.e.,
>   it cannot introduce the possibility of a Throwable that's not an Error
>   nor an Exception.

I.e. this proposal would require Joe-E programs to explicitly exit on a
VirtualMachineError, rather than modifying the VM to exit or (as in MarkM's
suggestion) requiring the VirtualMachineError to be uncaught.

Since Joe-E is a sequential language -- although an event-loop framework
for it has been mentioned -- terminating the VM and terminating the current
thread have a similar effect. However there might be a difference if a program
launcher used threads in a single Java VM to run multiple Joe-E programs.
Note that in that case, an uncaught exception in a thread will unlock monitors
held by the thread.

Once I had caught up with the discussion, I agreed that the VM or current
thread should terminate (short of making more extensive language changes such
as transaction support), and said that how it is implemented is a secondary
consideration. I'm not sure that this is correct now -- the fact that monitors
are unlocked seems problematic if there are any objects shared between threads
by the launcher implementation, or by Java's standard library.

(I also complained that the behaviour in E and Java of finally clauses that
throw exceptions, which had motivated MarkM's proposal to forbid them, is
clearly the Wrong Thing in general.)

In a later post, David Wagner suggested three other variants -- "Dave's proposals
#1 to #3" -- all of which would cause the thread in which a VirtualMachineError
(or Error for proposals #2 and #3) occurs to terminate as a result of an
uncaught exception, but without forbidding finally clauses.

MarkM also described a "less severe proposal" by Alan Karp:

> * Have Joe-E outlaw the catching of Errors.
> * Have Joe-E require all try-finally code to follow the following boilerplate:
>    try {
>        stuff1 // only normal Joe-E restrictions
>    } finally {
>        try {
>            stuff2 // no non-local exit via return, break, continue, or goto
>        } catch (Exception ex) {
>            throw new Error(ex)
>        }
>    }

Both I and David Wagner objected that this doesn't really solve the problem,
although I concentrated on defensive consistency (an attacker may have access
to an inconsistent object at 'stuff2'), and DavidW on access to non-determinism.
In any case, the Error thrown afterward is too late.

There was some discussion about whether only VirtualMachineErrors need to be
treated specially, or whether this is just a special case of a more general
problem with exceptions and non-local exits. David Wagner wrote:

> It seems to me that, while exceptions do pose a challenge for defensive
> consistency, the main issue with VirtualMachineErrors is not defensive
> consistency.  VirtualMachineErrors don't pose challenges to defensive
> consistency that go beyond the challenges already present with other
> kinds of runtime exceptions.  Rather, it seems to me that the unique
> challenge that VirtualMachineErrors present is the issue of access to
> non-determinism.

There's no doubt that VirtualMachineErrors are the most obvious and serious
case, because they can happen nondeterministically at any time. However I pointed
out that Java has many other exceptions that are thrown *implicitly* by operations
other than a 'throw' statement or method call, and that any method call can throw
a RuntimeException or Error.

Tyler also replied to David Wagner's post:
> The special issue with VirtualMachineErrors versus other exceptions is
> that the possibility of such Errors makes it impossible to write code
> that is guaranteed not to throw. For example, I can't write a private
> helper function that is trusted by the containing object to do some task
> without causing an exception. Just calling the helper function will
> allocate another stack frame, which may be one too many, resulting in a
> java.lang.StackOverflowError. VirtualMachineErrors make it too hard to
> implement the object's own internal logic.

I've missed out some stuff in this summary (especially from my own posts),
but I think it covers the main points.

David Hopwood <david.nospam.hopwood at blueyonder.co.uk>

More information about the e-lang mailing list