[e-lang] E / Python interoperability suggestions?

[Kevin Reid]

On Sep 1, 2010, at 5:57, Thomas Leonard wrote:

> What are the current thoughts on interoperability between E (captp)  
> and
> other languages / protocols?

Item: We want to use CapTP as an interop protocol. As mentioned below,  
this will require some revisions to CapTP. Depending on the target  
language, it might also involve more or less of building a mini E- 
style-references system inside the language -- Caja-CapTP is an  
example of such a system.

Item: We should also implement other message protocols for use from an  
E system, simply in the name of E being a general purpose platform.  
But there is no specific plans to do this.

> One option would be to implement captp / Pluribus in Python. It sounds
> like this might be difficult:
>
>  http://www.eros-os.org/pipermail/cap-talk/2007-February/007540.html

It is not so much difficult as nontrivial. Caja-CapTP is such an  
outside implementation of CapTP; it is written in JavaScript because  
we think that having a 'native' system is important for this  
particular use case. We hope that it will become the default comm  
system for Caja applications (or rather, not Caja, but the cap- 
JavaScript line of thought). Keyword: "Dr. SES" - Distributed Reliable  
Secure EcmaScript.

> There was also some interest from a Twisted Python developer:
>
>  http://www.eros-os.org/pipermail/e-lang/2005-April/010565.html
>
> MarkM said:
>
>        "Another option you should seriously consider is implementing
>        the web-calculus before-or-instead-of implementing CapTP.
>        CapTP/Data-E isn't really designed to be a good standard for
>        interoperation between cap systems, whereas the web-calculus  
> is.
>        I expect and hope to see a web-calculus comm system in E for E
>        eventually. CapTP will continue to be E's default, since it  
> more
>        transparently extends E's local semantics, but the web-calculus
>        is a better way for E to interoperate with other cap systems."

I'm not recalling precisely what web-calculus was in 2005 vs Tyler's  
current systems, but in general I would say that the protocols are  
different enough that you will want to have both options.

> There seem to be other "captp-inspired" protocols - such as
> http://code.google.com/p/caja-captp/ - but it's not clear whether this
> actually interoperates with E.

Caja-CapTP currently does not actually work -- it is missing the VatTP  
components and so it cannot actually communicate over the network. I  
hope to resume working on that this fall; starting with 'server'  
rather than 'browser' communication using node.js socket protocols.

It is my goal that Caja-CapTP will interoperate with all other  
'modern' CapTP systems, where 'modern' means after the transitions  
discussed below.

> About captp, MarkM said:
>
>        "There are four protocol transitions CapTP needs to make. Given
>        the
>        current minuscule adoption, I'm inclined to make all four  
> changes
>        simultaneously and not worry about compatibility.
>
>        * Switching from our custom DataComm to a VatTP layered on  
> top of TLS.
>        Tyler and Kevin have both made progress towards this goal.

This work, for any of the three CapTP implementations, has not been  
started. E-on-Java's CapTP still uses TLS. In my two new CapTP  
implementations, no VatTP layer exists yet.

>        * Switching from Java serialization, as used by E-on-Java, to  
> Data-E,
>        as used by E-on-CommonLisp and Caja-CapTP.

This is strictly E-on-Java implementation work. It should be just a  
matter of replacing all of the references to Java serialization  
(including the special read/replace objects) to Data-E; my CapTP-on-E  
should serve as a reference for doing so.

>        * Dropping the SwissHash from the live reference protocol as  
> discussed
>        in this thread.

I believe I have already done this at least once; this is minor work  
(that happens to be an incompatible protocol revision) that follows  
naturally from switching to the new Proxy interface (ProxyResolver  
being the old one) for implementing far refs.

>        * Change the exposed object APIs (esp the Miranda Methods) to
>        something more language neutral, as already adopted by Caja- 
> CapTP, so
>        that Caja-CapTP and E-CapTP can interoperate better."

We still need something to fill the role of the CapTP-related Miranda  
methods (__whenMoreResolved, __whenBroken). It is unclear to me what  
can be done instead; perhaps making those particular messages be out- 
of-band in the CapTP protocol but joining them in E implementations.

There is another transition: Adjusting Data-E so that the core  
semantics do not assume E's particular set of  
"literal"/"atomic"/"leaf" objects. Caja-CapTP implements this; Data-E  
builders instead of having buildLiteral/1 (which accepts exactly  
any[String, float64, int, char]) they have buildAtom/1 for a set of  
values specified by the guard returned by atomType/0. This also means  
that Data-E kits can provide efficient support for sending, for  
example, byte arrays; something which is completely lacking in the  
current Data-E.

Another thing is that we'd like to do this sometime:
http://wiki.erights.org/wiki/Ad-hoc_serialization_comity
But that is an additional refinement and there are higher priorities,  
like having actual CapTP interop.



I really, really, really need to get back to working on this, but I  
haven't gotten in the right mindset to work on this in my spare time  
outside of school.

If anyone would like to help...

-- 
Kevin Reid                                  <http://switchb.org/kpreid/>