[e-lang] Control flow considered harmful was: Proposal: Sugar-freewhens

[Dean Tribble]

Andreas Raab wrote:

> Very interesting, thank you so much! I will need some more time to 
> chew on this (and likely come back with more questions) but one thing 
> that strikes me in your argumentation is that (naively speaking) it 
> looks as if these arguments all apply to a distributed system where 
> certain properties (such as full order, or atomicity) are hard or 
> impossible to achieve. Would you agree with this statement?

Just getting caught up from vacation.  Thanks for the interest!

I would not entirely agree with the statement at two levels: 

DEPENDENCE, ATOMICITY and to some extent DEADLOCK are all properties 
that emerge in single-threaded applications.  The common problem of 
manipulating a collection while iterating it, for example, is a problem 
of reentrancy to the collection, which is an atomicity problem 
(copy-on-write or queuing events for the operations on the collection 
both work around this problem).  DEADLOCK is just the flip-side of 
ATOMICITY.  If an object does *not* allow reentrance, then you have the 
equivalent of deadlock in a single-threaded system.  For example, some 
collection implementations throw an exception when trying to operate on 
a collection being iterated; that's the equivalent of emergent deadlock.

Ignoring the problems that arise even in simple sequential systems, then 
I'm definitely focused on systems with the properties you describe.  
Lest that get you unduly excited about tea-time, virtual syncrony, etc., 
as we discussed at the Croquet workshop, systems that simulate global 
time require strong vulnerability/trust between the nodes.  That is 
inherent in all such systems invented so far, and is likely inherent in 
the requirements of that model.  Such systems would therefore not be 
suitable for the kinds of applications that I care about in which 
different interests are cooperating across the network.  When crossing 
trust-boundaries, it appears that you cannot hide the non-atomic, 
partially-ordered nature of the universe :-)

Note that this is what lead to my suggesting that tea-time style 
approaches be used for replication and reliability of Vats.  The 
inter-vat world would still reflect the partially-ordered, non-atomic 
nature of network systems, but each Vat would be multiply hosted and 
highly available.

> ----- Original Message ----- From: "Dean Tribble" <tribble at e-dean.com>
> Sent: Wednesday, December 22, 2004 10:46 PM
>
>>>> When-chaining is essentially open-coding continuation-passing 
>>>> control-flow. to achieve synchronous call.  It didn't make actors 
>>>> able to succeed.  It doesn't work at a system level.  It doesn't 
>>>> work no matter how much programmers new to distributed computing 
>>>> want it to.  It doesn't work no matter how nice the syntax is for it.
>>>>
>>>> Dataflow works.
>>>> </rant>
>>>
>>>
>>> Say more about this. It's the first time I've heard this 
>>> argumentation and clearly, given your strong feeling about it you've 
>>> got some experience with both styles. I'm curious what exactly the 
>>> problems at the system level are.
>>
>> <rant>
>>
>> Many of the systems, models, and alternatives explored before and 
>> during the Joule effort involved variations of this sort.  Rather 
>> than get too worked up, I'll just visit a bunch of different kinds of 
>> issues that show up, and you can ask for expansion on ones that are 
>> not obvious in retrospect.
>>
>> IMPLICIT FULL ORDER:  Control-flow ordering essentially imposes a 
>> total order on a partially ordered set of statements.  You cannot 
>> tell what of that imposed order is important, and what is 
>> incidental.  For example, Marcs's and my sequential account examples 
>> were the same, whereas our eventual variations differed in behavior 
>> under error.  That's because there is an implicit coupling dependency 
>> on following statements that the statement before them succeeded.  
>> His service would only get invoked if funds were available.  Did he 
>> intend that?  It matters a lot more when you are crossing security 
>> boundaries.  This might argue for considering an error construct that 
>> did *not* modify flow of control even in the sequential case.
>>
>> PARTIAL ORDER: Distributed systems are partially ordered.  One of the 
>> biggest and most frustrating values of Joule is that you could not 
>> get away with anything.  Any assumption that the world was fully 
>> ordered almost immeidately came back and bit you.  I was astonished 
>> at the number of times I was assuming more order than was actually 
>> present in a distributed system.  For example, if A send to B then C, 
>> and they both send to D, what assurance do I have that B and C's 
>> events on D are ordered in a particular way?  In the sequential 
>> world, I can be confident.  In the distributed world, any model I 
>> have is likely to be wrong.  Unless of course, each layer is 
>> carefully making sure to keep everythign fully ordered.  Then see 
>> SNOWBALLING and DEADLOCK
>>
>> ATOMICITY LOSS:  In the trivial examples examined so far, there is 
>> not an boject, and so there are no other side-effects.  That's simply 
>> not realistic.  By moving code into a when, you've lost the atomicity 
>> that enables simple coordination in a chaotic world.  For example, if 
>> you need to atomically update your object state to spend the money 
>> and get the result of the service, then the service invocation cannot 
>> be in the nested when clause with the option activation, because 
>> otherwise another event could get in there and change the object, its 
>> account, the service, etc. The simple atomicity in E of being able to 
>> do things and put your invariants back to right before taking another 
>> external event is a huge source of power, simplicity, and robustness.
>>
>> DEADLOCK: everything in a when block is "waiting" for the event that 
>> calls back to that when.  By using that for control-flow ordering, 
>> you essentially map the control-flow restrictions into dataflow.  
>> That simply means you bring all the deadlock problems into the 
>> dataflow world, so emergent deadlock starts to appear again.  The 
>> observer example in MarkM's CML response is of that flavor.
>>
>> SNOWBALLING: the ordering trick kinda works within a method.  How do 
>> you ensure that operations expressed byt he caller of a method happen 
>> *after* the actions (now embedded in when clauses) that are spawned 
>> by the method. Well, you can pass in a thunk, or always pass back 
>> just the right promise and have the caller do a when on that.  So 
>> which libraries should we make follow that pattern?  You pretty 
>> quickly need most of your libraries to follow the same pattern, or 
>> they cannot be used in the distributed context.  Then, you are 
>> programming in the actors model, in which most returns are via 
>> callbacks rather than simple return.  That's a much harder model to 
>> understand.  Similarly, all the services that you call need to work 
>> together to implementation the IMPLICIT FULL ORDER so that the natral 
>> PARTIAL ORDER of a distributed system doesn't surprise you.
>>
>> INHERITANCE:  The same pattern issue arises in trying to use 
>> inheritance or other object patterns.  You can no loner reliably call 
>> your superclass; you start needing to eventually send to your 
>> superclass, with all the additional coordination overhead.
>>
>> DEPENDENCE:  Since your forward computation is now dependent on the 
>> resolution of results from a call, your termination becomes dependent 
>> on the return of another service.  That fine within a single vat 
>> (because it could just infinite loop), but not fine between 
>> services.  Thus, you need to avoid such dependencies.  They lead to 
>> brittleness and cascading failures.
>