[E-Lang] Thoughts on Persistence in E.

[Mark S. Miller]

At 08:26 AM Wednesday 9/19/01, steve jenson wrote:
>Notes based on this and other experiment:
>
>1) I can successfully unserialize only strings and ints. 

Actually, you can serialize many more things than that, but far from enough. 
Over the next releases, I'll be expanding the set of things known to 
serialize correctly.  Everything that passes "Ref isPersistent(..)"
http://www.erights.org/javadoc/org/erights/e/elib/ref/Ref.html#isPersistent(java.lang.Object) 
is supposed to serialize correctly.  The set which I believe currently does is:

Broken references
null
arrays
FarRefs and promises, according to the javadoc at that URL.

All the HONORARY Persistent types should persist.  These are:

* All scalars -- from the E language's point of view:
    boolean
    char
    integer
    float64

* String
    Though this is not meaningful as a separate entry from the E 
    programmer's point of view.  To the E programmer, a String is a kind of 
    ConstList, and so persists because ConstList implements Persistent, below.

* java.security.Key & java.security.KeyPair

* Throwable (though, in a JDK <= 1.4, the java stack trace is lost)

* SturdyRef
    Oops on this last.  I've reproduced your NullPointerException (that 
    Elmer transcript was very useful), and will be fixing this.  In the 
    meantime, as you've noticed, you can just use the cap: uri string.


The classes of general interest that implement Persistent, that I believe do 
persist and revive correctly at the present time are:

    NetConfig http://www.erights.org/javadoc/net/vattp/data/NetConfig.html
    StaticMaker, which, to the E language programmer, means the object that 
        results from importing a Java class.  The object whose methods 
        correspond to the static method of the Java class.
    Brand, Sealer, Unsealer, SealedBox
    StemCell, RemoteCall, RemoteDelivery
        Use to implement both pass-by-construction and revive-by-construction.
        Needs to be covered in a separate mail message.

    All the variety of ELists (both FlexLists and ConstLists, including Strings)
    All the variety of ConstMaps.
    (The FlexMaps don't persist yet, and it may be a long time if ever before
      WeakKeyMap and WeakValueMap persist.)


>0) I can serialize anything without an error being spit out.

Yes, I tentatively made a strange decision here.  If I encounter something 
that's not serializable, I serialize a broken reference rather than throwing 
an exception.  This was, a bad subgraph cannot prevent the persistence and 
revival of the overall graph.  I feel very uncertain about this decision, 
and it's certainly worth arguing about.

For the corresponding issue in CapTP I made opposite decision: I throw an 
exception, causing the serialization to fail.



>2) While it appears that I can successfully deserialize a SturdyRef, when
>   I attempt to call liveRef() on that SturdyRef, I get a 
>   NullPointerException.

A real bug.  Thanks for catching it.  I will fix.


>3) I cannot Unserialize class prototypes nor instantiations of classes. The
>   value string spat out by Unserializer when I unserialize these is:
>  #value: [<ref broken by problem: not persistent: <queue>>, <ref broken by problem: not persistent: <queueMaker>>

What do you mean by "class prototype"?

Also, objects defined in E do not yet persist, and they won't at least until 
we start using Dean's transformer.  Dean's transformer also gives the 
objects a representation that would be reasonable to save and restore.  The 
current E interpreter does not.


>4) I cannot Unserialize functions which isn't shocking, as they are really
>   objects under the hood.

Right.  Same as case #3.


>E's persistence is new, I'm sure MarkM and gang are still smoothing out
>all of the persistence qualities. I personally really like how you can 
>keep objects together in a contextually close manner using Lists,
>the only thing that would make it better is that instead of having to
>explicitly remember the order in which things are Serialized, we could
>pull them out by key, just like Python's 'shelve'.

I don't know "shelve", but ComstMaps should work now, and it sounds like 
they might do the trick you're looking for.


        Cheers,
        --MarkM