[cap-talk] Handling upgrade with the Waterken Server (was: A paper on web-keys)

[Tyler Close]

Hi Pierre,

On Sat, Mar 22, 2008 at 8:06 PM, Pierre THIERRY
<nowhere.man at levallois.eu.org> wrote:
> Scribit Tyler Close dies 22/03/2008 hora 13:44:
>  > Feedback appreciated.
>
>  Having worked with persistence and the upgrade (open) problem in Common
>  Lisp, I find it a bit disturbing that you'd need to stop the application
>  and restart it to be able to upgrade it. It's also a bit strange that
>  you can't reclaim storage without destructing the underlying vat.
>
>  I suppose the first is inherent to the compile-and-run nature of Java,

I probably could rig up something to enable upgrade without restarting
the JVM. The trickiest bit would be noticing when the code had
changed.

>  but isn't it possible to leverage Java's GC to also collect persistent
>  garbage?

No, and even if you could it wouldn't help.

The no comes from the fact that Java's GC doesn't know about all the
objects. I'm careful to never load objects that aren't used.

Since all of the persistent state is, by definition, referenced
directly, or indirectly, by exported web-keys, a local GC algorithm
would only tell you there's not much to delete. There might be some
stuff to delete if an indirectly referenced object is no longer
referenced; however, I suspect the vast majority of the state will be
directly referenced by a web-key. So unless you want to get into
distributed garbage collection, the only option is explicit
destruction based on application-level logic. I suspect explicit
destruction at vat-level granularity is easier to manage than explicit
destruction at the object level.

>  Also, I'm not sure I understand how you deal with multiple upgrades on
>  objects you didn't need to access. If at least one of the previous
>  upgrades needed an algorithmic step, you need to write a method for it,
>  but IIUC, the method only exist in transient bytecode, so when you
>  restart the application with the new version of the code, you lose the
>  previous method.
>
>  Does that mean you need to ensure that every object will be deserialized
>  to be upgraded?

Yup, that's what the touch.jar command does:

http://waterken.sourceforge.net/upgrade/#touch

> What happens if the applications stops while
>  deserializing, before having upgraded every object?

The transaction doesn't commit and you're left back in the pre-touch state.

>  I think that concrete examples with fully runnable code would be a great
>  help to properly grasp the techniques you describe.

So each section has a "For example, ..." part, with either inline
code, or references to running code from the subversion repository.
Could you be more specific about how these examples are insufficient?

Thanks,
--Tyler