[E-Lang] Draft Kernel-E DTD & Sketch of translation to debuggable Java

[Mark S. Miller]

At 12:24 PM 9/27/00 , Dan Bornstein wrote:
>After having spent a large fraction of the last few months dealing with a
>large distributed app that uses RMI for most of the IPC, I'll have to weigh
>in that I have concerns about JOSS.
>
>The short story is: If all you ever expect to send over the wire are
>instances of classes that are already known to both ends (and are serialver
>equivalent on both sides), then you're okay, but if you *ever* expect to
>have one end be able to receive instances of a class it didn't already know
>about, or receive instances of a class that are of a newer version than the
>one it knew about, then forget about it. JOSS doesn't seem to provide any
>particular support for that sort of thing. (The RMI layer sortakinda
>tries, but it ends up falling pretty flat too.)

It seems there are two cases: the non-mobile-code-supporting case, and the 
mobile-code-supporting case.  For the first, it seems JOSS already does the 
right thing: the unserialization fails immediately upon encountering the an 
instance of the unknown class.  What else could it, or anything else, do?

For the mobile-code supporting case, here's my approximate plan:

Have each side of a connection maintain a table of the serialvers of the 
classes it believes the other side already knows how to unserialize.  This 
table would normally be initialized with the serialvers of all the standard 
E pass-by-copy classes (but perhaps could be negotiated to something else).  
When serializing a pass-by-copy object that's an instance of a class in the 
table, no problem, just do the normal thing.  If the class isn't in the 
table and this class isn't the result of compiling E to Java, then fail, 
since only E code is mobile.

If the object's behavior is defined in E, then serialize its Kernel-E parse 
tree into a packet to be transmitted ahead of this one, and remember its 
serialver in the table.  The receiving side will receive, audit, and compile 
to Java this Kernel-E parse tree before receiving the packet containing the 
instance of this parse tree.  Since the receiving vat's serialver for this 
Java class should be the same as the sending vat's, the instance should 
unserialize just fine.

Eventually, we might like to make the transmission of the parse-trees be on 
an as-needed basis, whether by asking the sending vat or by a 
repository-like mechanism, but this interacts badly with preserving the 
partial ordering constraints: the receiving vat would have to hold back 
further incoming messages until its need for the old parse-tree resolved.


>If it were me designing the wire protocol (and I know it's not), then I'd
>do something more like design the XML DTD first, having it contain things
>directly meaningful, and then construct the mapping from that DTD to
>appropriate Java objects, which would have to be instances of globally
>agreed-upon immutable-forever (at least in terms of their serialization
>format) classes. The JOSS format would be the happenstance of how those
>objects serialized, rather than the essential thing from which the other
>formats would be derived. 

JOSS is well defined.  I don't understand what the "happenstance" above, or 
the "whatever a two element Vector consisting of an Integer and a String 
happens to serialize into" below is meant to imply.  Isn't your proposed XML 
encoding equally the happenstance of how you (or hypothetically, I) decode 
to encode a vector of and integer and a string?

In any case, I don't see how this deals with the problem you originally 
raised: What to do when one side serializes an a pass-by-copy object which 
is an instance of a behavior the other side has never heard of?


>I'd say this arrangement would be a good thing,
>since raw JOSS is pretty opaque (even when interpreted into human-readable
>text) and seems a bit space on the space-inefficient side, especially given
>that it's a binary format. (Actually, I wouldn't be surprised if many XML
>expressions constructed this way ended up taking up fewer bytes than the
>equivalent JOSS.)

Yeah.  It's not great.  But as Arturo rubbed my nose in, it has adequate 
support for per-object upgrade (not ideal, but much better than my first two 
attempts at serialization, and better than XML).  And most of all, they are 
the only way within the Pure Java rules to make use of their native methods 
for reaching into an object's instance variables efficiently.  But given the 
binary-verbosity you and Alan point out, it would be good to know if there's 
a net efficiency win in the end.


>Simple example: Let's say you had strings, ints, and vectors as types
>you wanted to be able to directly represent, you might end up with
>XML like this:
>
>     <vector>
>       <el><int>1</int></el>
>       <el><string>foo</string></el>
>     </vector>

Ok, I give up.  I'm new to this XML DTD design thing, so help me out here.  
Why didn't you say:

     <vector>
       <int>1</int>
       <string>foo</string>
     </vector>

AFAI can tell, the latter satisfies all the XML desire for explicitness.  
Surely, even XML advocates don't advocate verbosity for its own sake?



         Cheers,
         --MarkM