More ProxyComm issues: VLS-free operation?

Eric Messick eric@syzygy.com
Sat, 26 Dec 1998 00:51:20 -0800


This stuff looks like it falls in my department.  I haven't looked at
the code in it's current form, but I can speak from the past.  Sounds
like there haven't been any substantial changes.

Point 5, below, is the stickiest issue, and it's the motivation for
some of the other (otherwise apparently odd) behaviors.

What it comes down to is this: If you don't want to run a VLS, then
you have to place your Vat at a well known location in order for it to
be findable.  What do I mean by "well known location"?  Basically, you
have to specify the port at which the Vat will listen.  The old
Registrar code had a constructor that took the listenAddr(ess) as an
argument.  A listenAddr was a string containing an IP address or DNS
name and a port number.  If you specify this, then you can construct a
valid search path before the listen() call completes.

In ordinary operation the key events go as follows:

Registrar is created, including a search path of VLS's.

A ListenThread is created.

Before the ListenThread has a chance to run, some SturdyRef's might be
constructed.  These need to know that search path from above.

Eventually, the ListenThread runs, does a listen() asking the OS to
allocate a new unused port to listen on, and finds out what port it
got.

The ListenThread touches off a chain of events (in yet another thread,
as I recall) telling all of the VLS's in the search path what the real
listenAddr is.

It's not until this point that another Vat can succeed at contacting
this one.

See more comments interspersed below:

In message <4.1.19981225114040.009d9960@ricochet.net>, "Mark S. Miller" <markm@caplet.com> wrote:
>I believe the proxy comm system has been designed to allow operation without 
>any running Vat Location Services (VLSs).  Indeed, with my current meager 
>understanding of this code, I think I see mechanism attempting to achieve 
>this, but it doesn't seem to succeed.  It seems the problems are confusions 
>about search paths.
>
>I'm not yet trying to create multiple vats in one jvm, as I await
>resolution of the magic powers vs multiple instantiation issue (see previous 
>note "proxy-comm issues.").  Currently I'm trying the simplest 
>two-vat/two-jvm scenario.  I'm running both processes on my laptop while not 
>connected to any network.  Since I normally do the SLIP/PPP thing, while I'm 
>running disconnected I'm simply localhost/127.0.0.1, but this should still 
>allow one local process to connect to another.
>
>
>In EEnvironment.java, there is:
>
>    /* Make sure there's a search path to us */
>    String flattenedSearchPath = props.getProperty("e.SearchPath",
>                //XXX Default is a hack for testing under Cafe
>                "localhost:4567;localhost:4568");
>    if (flattenedSearchPath == null || flattenedSearchPath.equals("")) {
>       throw new RuntimeException("'e.SearchPath' property notspecified");
>    }
>
>
>and in ConnectionsManager.java, there is
>
>
> /*package*/ ConnectionsManager(KeyPair identityKeys, 
>                                String /*nullOK*/ listenAddress,
>                                Runner runner)
> {
>     myIdentityKeys = identityKeys;
>     myPLS = null;
>     myLocalVatID = VatIdentity.calculateVatID(myIdentityKeys.getPublic());
>        
>     //XXX should use the e properties object (as held by the EEnvironment)
>     myLocalFlattenedSearchPath = System.getProperty("SearchPath",
>                 //XXX Default is a hack for testing under Cafe
>                 "localhost:4567;localhost:4568");
>     if (myLocalFlattenedSearchPath == null 
>                || myLocalFlattenedSearchPath.equals("")) {
>         Trace.comm.errorm(
>            "You must specify the SearchPath property to create a "
>            + "ConnectionsManager");
>         Trace.comm.notifyFatal();
>     }
>
>
>1) I added one of the above "//XXX Default ... localhost:4568" cases above, 
>by analogy with the other one, since the following tests are unhappy when 
>there's no search path.
>
>
>2) Tracing a warning if there's no search path would be fine, but tracing an 
>error or throwing an exception seems excessive, as this is reasonable for 
>completely vls-free operation, which we should allow.

The issue here is not quite vls or no vls.  As explained above, you
can have a valid search path with no vls.  Creating a
ConnectionsManager (or Registrar) implies that you want to be
contactable.  If you don't have a search path, you can't be contacted,
so the ConnectionsManager will be inneffectual, hence the error at
its creation.

Perhaps what's missing is a simple way to start a program that has no
intention of ever being contactable.  Such a program would not create
a ConnectionsManager, and thus not get the error.

If you want two Vats to be able to talk to each other, you have to
create a Registrar, and you have to give it a valid search path up
front (because of point 5).

>3) Chip's comment above "//XXX should use the e properties object (as held 
>by the EEnvironment)" is correct.  The ConnectionsManager constructors are 
>called only by each other or by the overloads of 
>VatIdentity.getConnectionsManager().  These are called only by each other or by
>
>    public ProxyManager(VatIdentity vat,
>                        String[] searchPath,
>                        RegistrationTable registrations,
>                        String listenAddress)
>    {
>        myVat = vat;
>        mySearchPath = searchPath;
>        myConnectionsManager = vat.getConnectionsManager(listenAddress);
>
>How about if this last call is also passes searchPath as an argument, 
>ultimately to be passed to the ConnectionsManager constructor?

I don't recognize ProxyManager, but it looks like this would be
correct.  The only question I would have would be where ProxyManager
gets created, and how does IT get the search path and listenAddress?

>     ******************  The Big One **********************
>
>4) Most interestingly, the code that's apparently trying to enable vls-free 
>operation is this in ConnectionManager:
>
>    /*package*/ void listingAt(String listenAddress) {
>        myListenAddress = listenAddress;
>        if (null != listenAddress) {
>            myLocalFlattenedSearchPath = listenAddress + ";" 
>                    + myLocalFlattenedSearchPath;
>        }
>    }
>
>which is called by the DataCommThunk, which is called by the ListenThread.  
>It is correctly adding the TCP/IP address of this vat to this vat's search 
>path, as known by the ConnectionsManager.  This would be fine if the search 
>path were maintained in one place and shared.  Unfortunately, there's also a 
>search path stored (at least) in the Registrar and the ProxyManager, and 
>neither of these are updated.  Should these instead obtain the search path 
>from the DataConnection?  Should there be an mutable search-path object 
>that (at least) all three of these share, so they all see the update?

There is some confusion here.  I think what's going on is that the
search path is used for two different (related) things.  First, it
goes into SturdyRef's.  Second, it's handed around as part of a three
party handoff.  In the first case, placing the actual listenAddress in
the search path doesn't make any sence.  In the second case it does.


>5) Actually, even if these were shared, we still have a race condition.  The 
>search path updating above happens later in the listen thread after 
>everything is allegedly operational.  If something, such as the following, 
>needs the search-path before the listen thread updates it, it could get a 
>search-path that doesn't include our own listenAddress.

But a three party handoff can't happen until the Registrar is OnTheAir
(and thus has a valid listenAddress).  It's only in this case that the
listenAddress is an important component of the search path.  If you've
specified the listenAddress, then it can be part of the static portion
of the search path.

>6) When you ask a Registrar to make a SturdyRef for an object, it uses the 
>mySearchPath in Registrar, which is never updated to include our own 
>listenAddress.
>
>
>7) Could whoever wrote the following amazing comment
>
>
>	/**
>	 *  "I have seen things you would not believe..."
>	 *
>	 *  @see ListenThread.startup()
>	 *  @see ListenThread.shutdown()
>	 */
>	/*package*/class UserThread extends Thread {
>
>please clarify?  Thanks.

Chip's clarification, while accurate, may not have been all that
helpful :-) I'll take a stab at it, though I didn't write the comment
in question.

The problem is that there's no way to tell the ListenThread to stop
listening.  It's sitting in the accept() system call which is
uninterruptable (at least under java 1.0.2).  It's possible that
accept() is now interruptable, in which case this horrible hack can be
disposed of with much rejoycing.

So, the ListenThread was turned into a daemon thread, allowing the
program to exit with ListenThread in accept().  For demo programs, it
would exit too quickly, because the demo was waiting only for outside
connections, which meant that the only active thread was the daemon
ListenThread, and so the jvm would exit.  To overcome this, the
UserThread was created.  It's sole job is to sit around and be
interruptable.  When you would otherwise have interrupted the
ListenThread, you interrupt the UserThread instead.  It exits, and
takes away a runnable thread.

>Any and all help would be greatly greatly appreciated!  Getting this all 
>working again soon would be a very big deal.  In the meantime, I'm about to 
>retreat to trying all this with VLSs.  I let y'all know how it goes.
>
>	Cheers,
>	--MarkM

I hope this was helpful...

-eric