[e-lang] Events as Bad Ideas

[Stiegler, Marc D]

Recently I had an email conversation with a fellow HP employee
interested in promise pipelined object capabilities (E) for his system.
I cc'd markm, and he thought folks would be interested in my critique of
a paper entitled "Why Events are a Bad Idea". So I have forwarded my
comments, along with the parts of the original email giving the paper
reference and context.

--marcs

> -----Original Message-----
> From: Stiegler, Marc D 
> Sent: Wednesday, November 24, 2004 9:59 AM
> Subject: Events as Bad Idea (was RE: A request for comments)
> 
> 
> A quick review and some quick comments:
> 
> The focus of "Why Events are a Bad Idea" seems to be 
> performance. The focus of E is robustness in the face of 
> large and evolving distributed systems; E promise pipelining 
> has as goals, high performance when latency is the limiting 
> factor, and an interface to the human programmer that 
> improves the programmer's chances of coming out alive. In the 
> programmer-interface goal, it is not unlike the goal that was 
> set by Djikstra's "Goto Considered Harmful": inside the 
> machine, goto branch instructions are the only thing you've 
> got for flow of control. But by wrapping these primitives in 
> for loops, if-then-else, and try-catch, you create something 
> humans can use, understand, and maintain.
> 
> Since coming to HP, markm and I have sat in cubes adjoining 
> people whose focus is massively parallel computers. We have 
> spent many lunch hours trying to figure out ways in which 
> promise pipelining might help them, but have so far always 
> concluded that, if you care about performance more than 
> everything else in the world, we do not have anything to 
> offer. I still feel that that is not correct, but so far, 
> anything else has eluded us. However, this is probably beside 
> the point, the massively parallel computers don't look to me 
> like the same problem as the Lustre problem.
> 
> The paper asserts, "...call/return, parallel calls, and 
> pipelines. All of these patterns can be expressed more 
> naturally with threads." E supports immediate calls for 
> call/return. I cannot imagine how a thread could do parallel 
> calls and pipelines more simply than E:
> 
>  a <- b()
>  a <- c(c <-d)
>  e <- g
> 
> Which does every kind of pipeline and parallel call I can think of. 
> 
> The real issue for robustness and programmer survival arises 
> when you start composing these things in a complex fashion. 
> Markm has this utterly delightful presentation in which he 
> starts out with an implementation of a call-return observer 
> pattern, shows the concurrency bug that causes JavaSoft to 
> recommend something different in the presence of threads for 
> Java Listeners (which are observers), shows a concurrency bug 
> with the JavaSoft canonical solution, shows the solution to 
> the new problem, shows a new concurrency bug this introduces, 
> and proves, quite powerfully at this point, "as you can see, 
> you can never be confident you have something that will 
> work." Then markm shows the solution in E, which is as simple 
> as the original implementation, and foolproof. Find the 
> slides at http://www.erights.org/talks/promises/index.html
> 
> "Common event patterns...map cleanly...onto threads...They 
> need a return event on the event model." We agree that a 
> return is needed, and this is where the promises come in. The 
> claim that common event patterns map cleanly onto threads is 
> just false if you look at systems complex enough to have 
> deadlock--a system susceptible to deadlock is not clean, 
> however easy the code is to read and misunderstand. Please 
> note, for a web server, this is not necessarily a problem. My 
> limited experience writing web servers suggests that you can 
> just spin off threads that will never talk to each other 
> again, and no deadlock threat looms. In this case, threads 
> might be only a little worse than promises--a little worse 
> because of the risk that one day you'll try to grow the 
> system into something that needs to do a little more 
> interaction, risking deadlock, in which case, if you're in 
> threads, you're out of position to stay robust.
> 
> I am skipping all the discussion about performance. They say 
> both systems can be made comparable in performance, I am 
> willing to believe this, and am not interested in arguing 
> about whether promise pipelining lures programmers into more 
> and better pipelining, minimizing latency, though I 
> personally believe it probably does.
> 
> Exception handling: E once again has a mechanism that was not 
> foreseen by the authors, the broken promise. This wipes out 
> their argument that threads are easier. "Fixing problems with 
> events is tantamount to switching to threads" is simply not true.
> 
> "supports use of threads for true concurrency". In a sense, 
> so does E: an E vat is a combination of a thread and an event 
> queue, where objects reside only in one vat/on one thread and 
> can only be reached via the event queue. We're still using 
> threads, we're just wrapping them up in a programmer friendly 
> interface, much as for loops package branch statements in a 
> programmer friendly fashion. Vats (and therefore threads) 
> have many valuable uses in E, so in this sense, I'd agree 
> threads are useful. Truthfully, given the improvements in 
> event handling presented by promise pipelining, I doubt that 
> many threads/vats are appropriate in web servers. I wrote one 
> partial web server in E. I found no reason to have more than 
> one explicit thread--though I don't know how many threads are 
> implicit in non-blocking disk IO, which is crucial if you 
> seek high performance in a web server. 
> 
> --marcs
> 
> > Hi, Marc.
> > 
> > I'm enjoying learning E. I'm also trying to design my own
> > "class project" as a way to grow what I learn via DonutLab 
> > into something of use in my own research. 
...
> > I have a request: It would help me if you would give your
> > opinion of the following paper and its relevance to E:
> > 
> >  http://www.enhyper.com/content/eventsbadidea.pdf
> > 
> > The paper's title is a swipe at Ousterhout's 1996 Usenix talk, "Why 
> > Threads are a Bad Idea (for most purposes)":
> > 
> >  
> http://www.stanford.edu/class/cs240/readings/threads-bad-usenix96.pdf
...
> > Anyway, if you're able to read & comment on the above 
> paper, I would 
> > really appreciate it.