[e-lang] E and FUSE-j integration

[Kevin Reid]

On Sep 26, 2007, at 7:11, Toby Murray wrote:

> Hi all on e-lang,
>
> I'm wondering whether anyone has thought about the possibility of  
> writing a FUSE (Linux Filesystem in Userspace) plugin in E that  
> would allow one to mount a captp URI in the local filesystem.

I've briefly thought of "this might be interesting and possible", in  
the context of implementing an automatic build system like the one  
described at
<http://cap-lore.com/CapTheory/Patterns/Cache.html>, that is provided  
as a filesystem. (I don't think that document is where I originally  
saw the idea, but I can't find a better one.)

> File operations on the filesystem would be implemented using E  
> remote method invocations.

There are two possible kinds of systems here:
   (a) present a filesystem layout which can express any E message.
   (b) expect the remote ref to implement a filesystem protocol.

(b) is more generally useful and can be used to imp

>
> (I think this would satisfy one of Jed's "Grand Visions/Challenges"  
> for capability security, amongst being nifty anyway.)
>
> FUSE-j provides FUSE bindings for Java. To implement a filesystem  
> in Java, one implements the 'FileSystem3' interface (no idea what  
> the significance of the '3' is though)
>
> public interface Filesystem3 extends FilesystemConstants
> {
>    public int getattr(String path, FuseGetattrSetter getattrSetter)
> throws FuseException;
>
>    public int readlink(String path, CharBuffer link) throws
> FuseException;
>
>    public int getdir(String path, FuseDirFiller dirFiller) throws
> FuseException;
>
>    public int mknod(String path, int mode, int rdev) throws
> FuseException;
>
>    public int mkdir(String path, int mode) throws FuseException;
>
>    public int unlink(String path) throws FuseException;
>
>    public int rmdir(String path) throws FuseException;
>
>    public int symlink(String from, String to) throws FuseException;
>
>    public int rename(String from, String to) throws FuseException;
>
>    public int link(String from, String to) throws FuseException;
>
>    public int chmod(String path, int mode) throws FuseException;
>
>    public int chown(String path, int uid, int gid) throws  
> FuseException;
>
>    public int truncate(String path, long size) throws FuseException;
>
>    public int utime(String path, int atime, int mtime) throws
> FuseException;
>
>    public int statfs(FuseStatfsSetter statfsSetter) throws
> FuseException;
>
>    // if open returns a filehandle by calling FuseOpenSetter.setFh()
> method, it will be passed to every method that supports 'fh' argument
>    public int open(String path, int flags, FuseOpenSetter openSetter)
> throws FuseException;
>
>    // fh is filehandle passed from open
>    public int read(String path, Object fh, ByteBuffer buf, long  
> offset)
> throws FuseException;
>
>    // fh is filehandle passed from open,
>    // isWritepage indicates that write was caused by a writepage
>    public int write(String path, Object fh, boolean isWritepage,
> ByteBuffer buf, long offset) throws FuseException;
>
>    // called on every filehandle close, fh is filehandle passed from
> open
>    public int flush(String path, Object fh) throws FuseException;
>
>    // called when last filehandle is closed, fh is filehandle passed
> from open
>    public int release(String path, Object fh, int flags) throws
> FuseException;
>
>    // Synchronize file contents, fh is filehandle passed from open,
>    // isDatasync indicates that only the user data should be flushed,
> not the meta data
>    public int fsync(String path, Object fh, boolean isDatasync) throws
> FuseException;
> }
>
>
> The naive idea would be to write an object in E that implements
> FileSystem3
>
> def myFileSystem {
>   to getattr(path, getattrSetter) {
>   }
>   // etc.
> }
>
> But there appears to be a fundamental problem here with E's  
> asynchronous remote communication model and the synchronous nature  
> of the FileSystem3 API. Suppose one writes a proxy to handle  
> mounting remote URIs (which would be the primary use of this thing  
> after all):
>
> def myFileSystemProxy {
>   to getattr(path, getattrSetter) {
>      remote <- getattr(path, getattrSetter)
>      when remote -> {
>        // it is only now that we know whether getattr has succeeded
>      } catch prob {
>        // getatttr failed, but how to notify FUSE since the  
> original call has already returned
>      }
>   }
> }

There's some confusion about the structure of sends and "when" here:  
the argument of "when" is a reference to wait for the resolution of.  
"remote" is (usually) already resolved, and not relevant to the  
*result* of the getattr send. You mean:

      def result := remote <- getattr(path, getattrSetter)
      when (result) -> {

or

      when (def result := remote <- getattr(path, getattrSetter)) -> {

> How would one go about trying to implement the synchronous FUSE API  
> using E's asynchronous remote messaging model?
>
> (I suspect the solution here would be a general one that could be  
> applied to other situations in which one needs to implement a  
> synchronous interface using remote references.)

Yes.

Firstly, the E vat must be in a separate thread+runner from the  
synchronous caller and callee. The callee holds a reference to this vat.

When the callee is invoked, it sends an appropriate message into the  
vat, and then blocks waiting for notification that the result promise  
has resolved.

There don't seem to be pretty operations for doing this; Vat#now/1  
only does one-turn operations. MarkM, is there an existing mechanism  
for this? If not, it shouldn't be hard to implement -- we just need a  
when-resolved-reactor which can wake up a thread, which we attach to  
the result promise.

> Also, another potential problem is that FUSE has its own event  
> loop. A call to mount a filesystem returns only when the filesystem  
> is unmounted. Hence, there appears to be conflict between two  
> competing event loops -- E's and FUSE's.

This isn't a problem with enough threads. :)

I'm not sure I understand -- do you mean that when you tell FUSE-j  
something like mount(path :String, _ :Filesystem3) it doesn't return  
until unmounted? This shouldn't be a problem; we just spin off this  
one operation into another thread -- which will presumably end up  
being the thread which does the deliver-and-block described above.

> Disclaimer: This is all a thought experiment at the moment. If  
> feasibility could be demonstrated (i.e. by coming up with a means  
> to address the above issues) then one might then think about a toy  
> implementation.

Have you heard whether FUSE-j is compatible with MacFUSE?

-- 
Kevin Reid                            <http://homepage.mac.com/kpreid/>