[e-lang] Sending files with E?

[Toby Murray]

On Mon, 2008-09-29 at 00:31 -0500, Jimmy Wylie Jr. wrote:
> Hi all,
> 
> I've been playing around with the persistent chat program provided by
> E in a Walnut towards the end.  I wanted to add to the program, the
> ability to send a file and receive files.  However, I'm not exactly
> sure how to do this.  I have written something, but it only works on
> small files, when I try to send something like a jpeg, I get a stack
> overflow error.

The overflow is likely being caused because the 'copy' function is tail
recursive, but E-on-Java doesn't implement tail call optimisation.

That is, the 'copy' function calls itself recursively for each byte of
the file. This means that the size of the runtime stack (the number of
stack frames) required to send an N byte file, is O(N). 

You can avoid this problem, however, whilst making minimal changes to
the code, by having 'copy' call itself recursively using the <-
operator, rather than using an immediate call.

Recall that calling a function like 'copy' as in

        copy(dispenser, output,    avail)

is equivalent to writing

        copy.run(dispenser, output,    avail)

You can avoid the stack overflow (I think) by changing the (implicit)
immediate call ('.') to a send ('<-') instead, e.g.

        copy <- run(dispenser, output,    avail)
> 
This will have the subsequent evaluation of 'copy' occur in a separate
turn. Rather than requiring another stack frame to evaluate it
immediately, it will be evaluated later on. Now sending an N byte file
requires a stack of some constant size -- i.e. a stack size of O(1).



> Anyone have any suggestions on how to improve this, or perhaps an
> easier way?  

There probably is an easier way. Your solution is functionally correct,
but it is certainly not the most efficient way to stream a file.

Kevin Reid has a new E IO library that might address this sort of thing,
but I'm yet to look at it. Kevin, any thoughts here?

> Also, I want both participants in the transfer to be notified of the
> transfer.  The copy method doesn't currently return a promise, but I
> know by using Ref.promise() I could create a [promise, resolver] pair,
> but I'm not sure if that will work either, since the copy method will
> keep recursively returning promises. Would that hog too much memory?  

You could create the promise before the first invocation of 'copy' and
pass the promise through to the call to 'copy'. On its last invocation
(i.e. if 'copy' is not invoking itself again), it simply resolves the
promise it was passed, to indicate that the transfer is complete.
Otherwise (on a recursive call), it passes the promise to the recursive
call without resolving it. Would that work?

Cheers

Toby