[e-lang] E and FUSE-j integration

[Toby Murray]

On Wed, 2007-09-26 at 09:20 -0400, Kevin Reid wrote:
> On Sep 26, 2007, at 7:11, Toby Murray wrote:
> 
> > Hi all on e-lang,
> >
> > I'm wondering whether anyone has thought about the possibility of  
> > writing a FUSE (Linux Filesystem in Userspace) plugin in E that  
> > would allow one to mount a captp URI in the local filesystem.
> 
> I've briefly thought of "this might be interesting and possible", in  
> the context of implementing an automatic build system like the one  
> described at
> <http://cap-lore.com/CapTheory/Patterns/Cache.html>, that is provided  
> as a filesystem. (I don't think that document is where I originally  
> saw the idea, but I can't find a better one.)
> 
> > File operations on the filesystem would be implemented using E  
> > remote method invocations.
> 
> There are two possible kinds of systems here:
>    (a) present a filesystem layout which can express any E message.
>    (b) expect the remote ref to implement a filesystem protocol.
> 
> (b) is more generally useful and can be used to imp

(b) is what I had in mind. (The sentence seems to have been cut off
prematurely btw.)

<snip>

> >
> > The naive idea would be to write an object in E that implements
> > FileSystem3
> >
> > def myFileSystem {
> >   to getattr(path, getattrSetter) {
> >   }
> >   // etc.
> > }
> >
> > But there appears to be a fundamental problem here with E's  
> > asynchronous remote communication model and the synchronous nature  
> > of the FileSystem3 API. Suppose one writes a proxy to handle  
> > mounting remote URIs (which would be the primary use of this thing  
> > after all):
> >
> > def myFileSystemProxy {
> >   to getattr(path, getattrSetter) {
> >      remote <- getattr(path, getattrSetter)
> >      when remote -> {
> >        // it is only now that we know whether getattr has succeeded
> >      } catch prob {
> >        // getatttr failed, but how to notify FUSE since the  
> > original call has already returned
> >      }
> >   }
> > }
> 
> There's some confusion about the structure of sends and "when" here:  
> the argument of "when" is a reference to wait for the resolution of.  
> "remote" is (usually) already resolved, and not relevant to the  
> *result* of the getattr send. You mean:
> 
>       def result := remote <- getattr(path, getattrSetter)
>       when (result) -> {
> 
Absolutely. Sorry 'bout that. Wasn't thinking very hard when I wrot ethe
above obviously.

> > How would one go about trying to implement the synchronous FUSE API  
> > using E's asynchronous remote messaging model?
> >
> > (I suspect the solution here would be a general one that could be  
> > applied to other situations in which one needs to implement a  
> > synchronous interface using remote references.)
> 
> Yes.
> 
> Firstly, the E vat must be in a separate thread+runner from the  
> synchronous caller and callee. The callee holds a reference to this vat.

So we have one thread that is running the caller. It invokes the callee.
There is another thread running an E vat. The callee is implemented in
Java (or whatever underlying language in which the E implementation is
written). It uses ELib or other library calls to asynchronously send a
message to the E vat and then blocks the calling thread, waiting for the
promise returned by sending the ascynrhonous message to be fulfilled.

The caller is now blocked and a message has been sent to the E vat which
is running in a separate thread.
 

The E vat services the message on the next appropriate turn, as usual.
At whatever point that the E vat has a result (which might be after it,
itself has had a promise to a remote invocation fulfilled), it fulfills
the promise on which the calling java thread is blocked.

This unblocks the calling thread, at which point  it has its result and
can return to the original caller.

The calling thread is then the FUSE event loop, right?

How does the E implementation prevent races between the calling (FUSE)
thread and the thread that's running the E vat? Or is this a non-issue
for some other reason?

<snip>

> Have you heard whether FUSE-j is compatible with MacFUSE?

Fuse-j implements the native (Linux) FUSE C API.

>From the MacFUSE page:
http://code.google.com/p/macfuse/

> Although MacFUSE has a completely different kernel-level
> implementation from Linux FUSE, it supports the FUSE specification
> well enough that many popular FUSE file systems can be easily compiled
> and work on Mac OS X--often out of the box. Examples of file systems
> that work and have been tested (to varying degrees) include sshfs,
> ntfs-3g (read/write NTFS), ftpfs (read/write FTP), wdfs (WebDAV),
> cryptofs, encfs, bindfs, unionfs, beaglefs (yes, including the entire
> Beagle paraphernalia), and so on.

Hence, one might be able to assume that the FUSE-j implementation of the
Linux  FUSE protocol could be compatible with the MacFUSE protocol. In
which case, FUSE-j might run as well on top of MacFUSE as it does on
Linux FUSE.

(I've never actually tried using FUSE-j, btw. It's only been something
I've b een looking into today.)