Build Problems Resolved
Paul Snively
psnively@earthlink.net
Thu, 2 Sep 1999 09:13:33 -0700
This is a multi-part message in MIME format.
------=_NextPart_000_0011_01BEF523.6E4C30A0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi Mark!
>You are forgiven ;)
Oh good!
=20
>In this last year, I believe we've achieved a sufficiently modular =
architecture that (with one exception noted below) there is no need to =
>start from scratch. E does not need a big monolithic rewrite. Rather, =
all the pieces of E can, and should, be rewritten over time. As >each =
replacement comes to be seen as superior to the corresponding existing =
component, the replacement can happen in-place. No >rewrite task need =
any longer to take on the E system as a whole.
I probably overstated my case: I didn't really imagine that E would =
need a TOTAL rewrite, although the thought that got me closest to the =
idea was admittedly the native-codegen idea, but we get to that a bit =
later on.
=20
>If this is true, then I have done my first job as open source =
coordinator: bringing the system to the point where other individuals =
can >incrementally improve the system, often by wholesale replacement of =
individual pieces, and without much coordination with work >happening =
elsewhere in the system. However, I hope you understand that I don't =
have the time to do all the required replacement >myself. It would be =
wonderful if some of the desire for a rewrite turned into a desire to do =
some of the rewriting ;) This would bring me >to my second job as =
open-source coordinator -- integrating in such submitted replacements.
=20
I'm keenly interested; the only question is what form the =
involvement takes. I have to confess that, while I certainly agree that =
the JVM is one legitimate target for E, I'm a bit unhappy with the =
notion that it's the *only* target, and putting E atop a new platform =
such as EROS must await Java on that platform--and not merely Embedded =
Java or even Personal Java (else porting Kaffe would solve the problem), =
but full-bore Standard Java. Otherwise I'd likely pursue some kind of =
native code generation strategy for E and try to get that and an =
appropriate runtime on EROS.
=20
> 1) Reintroducing persistence--of particular significance, =
perhaps, now that ColdStore is in beta.
>
>This is probably the element on your list with the greatest risk of =
requiring surprising non-modular changes elsewhere in E. We are not =
>currently planning to use ColdStore or any other native solution. As =
for what we are planning, Tyler, would you care to explain?
I suspect this is riskiest too, although hopefully that's mitigated =
somewhat by E's intention to have been an orthogonally persistent =
language from the outset. I could have sworn that ColdStore was under =
active consideration from what I've read on the mailing list archives, =
but I'm happy to be brought up to date as to what people are thinking. =
Again, IMHO, I'm a bit concerned with what seems to me to be a kind of =
"anti-native bias" on the project--why not have an E compiler just as =
there's a gcc compiler?
=20
=20
>2) Replacing the non-open-source OROMatcher (which, frankly, I =
suspect to be at the root of at least the Visual Age problem if not also =
the JIT >problem) with the GNU Java regex package.
>
>Replacing OROMatcher with another jvm-based regex package should be =
as modular as putting OROMatcher in was in the first >place. Where do I =
find the GNU Java regex package? Does it do Perl5 regular expressions =
(not a requirement, but it would be >good)?
=20
=20
<http://www.cacas.org/java/gnu/regexp/>. It apparently doesn't =
full support Perl5 syntax, but comes close enough for most purposes.
> 3) Replacing the rather byzantine byaccj process with a =
grammar etc. built around ANTLR. Using the tree support in ANTLR might =
lead us >directly to:
>
>Parsing the string of characters into a parse-tree should be, and I =
believe is, orthogonal from how/whether one generates code from t>hose =
trees. I would love for someone to replace the byaccj parser with an =
ANTLR-based parser, and I believe the result could be slid >in without =
effecting much else. Separately, I have been planning to generate jvm =
bytecodes from my parse-tree for a while now. I'm >perfectly receptive =
to the notion that ANTLR's tree-grammar stuff might provide a better =
framework for doing so. Again, I don't see the >change as effecting =
much elsewhere.
=20
=20
Right on all counts, of course; the point that I was trying to =
make--poorly--was that having ANTLR's tree-based transformation =
capability might ease the burden of doing both bytecode and native =
codegen.
>Btw, a more valuable exercise would be to integrate in ANTLR as a =
quasi-parser generation tool, such that its use to create an E->grammar =
quasi-parser was just one case of that. Were this done, then one should =
be able to turn any existing ANTLR grammar into a >quasi-parser for that =
grammar easily.
=20
Sounds like fun! Is there documentation on the quasi-parsing process =
that I should be aware of?
=20
=20
>4) Bytecode-compiling E rather than interpreting. Apart from =
leveraging ANTLR for this, we might also want to look at the open-source =
GNU Java >bytecode-generation library written by Per Bothner at Cygnus =
as part of his Kawa implementation of Scheme in Java.
>
>Worth a look. Please gimme a pointer.
=20
<http://www.gnu.org/software/kawa/>. Kawa actually provides a few =
packages--gnu.bytecode, gnu.expr, and gnu.mapping in particular--that =
might be of use in creating a bytecode compiler for E.
=20
=20
> 5) While we're compiling E, let's add support for multiple =
targets: Java bytecode + ELib being the first, of course, and some sort =
of runtime >(ColdStore etc.?) atop EROS being the second.
=20
>The E *language* itself could easily be retargetted to a non-jvm =
platform such as Squeak, or perhaps ColdStore. After all, E is an =
>almost minimal lambda language. However, there is the huge matter of =
the libraries. E benefits tremendously by being able to inherit >the =
Java libraries and provide them to the E programmer almost as if they =
had been written for E. Other platforms such as Squeak or >Python also =
have rich libraries it would be good to inherit, but one must choose =
which of these provide the default definition of the >standard E =
libraries. For various reasons, we've chosen Java's libraries.
>
>Having made this choice, it would be too much work to build =
compatible libraries on platforms in which the Java libraries were not =
>available as a starting point. Notice that this does *not* pin E into =
compiling to jvm bytecodes. In a system such as Cygnus's, where >jvm =
code is compiled to the runtime formats compatible with corresponding =
C++ code, and where they've already done the work to >reproduce the java =
libraries, we could compile into a native form that used these =
libraries.
=20
=20
Have they in fact reproduced the Java libraries? Has anyone tried to =
build E with, say, GCC 2.95.1's Java tools and Cygnus' runtime?
>This was as good as I thought it could get until Tyler made a =
crucial point: A major motivation for E is mobile confined code, =
especially >for Smart Contracting. A confined Smart Contract does not =
need, and typically will not be given access to, many of these Java =
>libraries. The normal environment for a confined Smart Contract will =
include important preexisting libraries, but probably a small >number, =
and probably not inherited from Javasoft's work. Once we've carefully =
defined this environment, we will probably find that we >can easily =
reproduce it on other platforms. =20
>
>So E isn't easily portable to non-java platforms, but the =
Confined-E subset of E should be.
=20
Hmmm. It'd be interesting to see what could happen given, e.g. the =
CLASSPATH project (cleanroom Java libraries) and Cygnus' Java-to-native =
compilation.
=20
>I'd love to see E running on EROS, but this should simply follow =
from a port of the underlying implementation platform (such as a jvm) to =
>EROS. Once E is on EROS, EROS's capabilities can be made to show up in =
E as E capabilities. I fantasize about E being EROS's >shell language.
=20
OK. So who's doing Java for EROS?
Anyway, despite having the sense that we're, to one degree or =
another, at philosophical odds as to what E *should* be, I greatly =
appreciate the information. It gives me a much stronger sense of why E =
is what it is and what's going to have to happen in order to accomplish =
some of its goals.
=20
=20
Cheers,
--MarkM
=20
Regards,
Paul
------=_NextPart_000_0011_01BEF523.6E4C30A0
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD W3 HTML//EN">
Hi Mark!
>You=20
are forgiven ;)
Oh=20
good!
>In this last year, I believe we've achieved a =
sufficiently=20
modular architecture that (with one exception noted below) there is =
no need=20
to >start from scratch. E does not need a big monolithic=20
rewrite. Rather, all the pieces of E can, and should, be =
rewritten=20
over time. As >each replacement comes to be seen as =
superior to the=20
corresponding existing component, the replacement can happen =
in-place. =20
No >rewrite task need any longer to take on the E system as a=20
whole.
I=20
probably overstated my case: I didn't really imagine that E would =
need a=20
TOTAL rewrite, although the thought that got me closest to the idea =
was=20
admittedly the native-codegen idea, but we get to that a bit later=20
on.
>If this is true, then I have done my first job as =
open source=20
coordinator: bringing the system to the point where other =
individuals can=20
>incrementally improve the system, often by wholesale replacement =
of=20
individual pieces, and without much coordination with work =
>happening=20
elsewhere in the system. However, I hope you understand that I =
don't=20
have the time to do all the required replacement >myself. =
It would=20
be wonderful if some of the desire for a rewrite turned into a =
desire to do=20
some of the rewriting ;) This would bring me >to my second =
job as=20
open-source coordinator -- integrating in such submitted=20
replacements.
I'm=20
keenly interested; the only question is what form the involvement =
takes. I=20
have to confess that, while I certainly agree that the JVM is one =
legitimate=20
target for E, I'm a bit unhappy with the notion that it's the *only* =
target,=20
and putting E atop a new platform such as EROS must await Java on =
that=20
platform--and not merely Embedded Java or even Personal Java (else =
porting=20
Kaffe would solve the problem), but full-bore Standard Java. =
Otherwise I'd=20
likely pursue some kind of native code generation strategy for E and =
try to=20
get that and an appropriate runtime on EROS.
> 1) Reintroducing=20
persistence--of particular significance, perhaps, now that =
ColdStore is=20
in beta.
>
>This is probably the =
element on your=20
list with the greatest risk of requiring surprising non-modular =
changes=20
elsewhere in E. We are not >currently planning to use =
ColdStore or=20
any other native solution. As for what we are planning, Tyler, =
would=20
you care to explain?
I=20
suspect this is riskiest too, although hopefully that's mitigated =
somewhat=20
by E's intention to have been an orthogonally persistent language =
from the=20
outset. I could have sworn that ColdStore was under active =
consideration=20
from what I've read on the mailing list archives, but I'm happy to =
be=20
brought up to date as to what people are thinking. Again, IMHO, I'm =
a bit=20
concerned with what seems to me to be a kind of "anti-native =
bias"=20
on the project--why not have an E compiler just as there's a gcc=20
compiler?
>2) Replacing the non-open-source=20
OROMatcher (which, frankly, I suspect to be at the root of at =
least the=20
Visual Age problem if not also the JIT >problem) with the GNU =
Java=20
regex package.
>
>Replacing =
OROMatcher with=20
another jvm-based regex package should be as modular as putting =
OROMatcher=20
in was in the first >place. Where do I find the GNU Java =
regex=20
package? Does it do Perl5 regular expressions (not a =
requirement, but=20
it would be >good)?
<">http://www.cacas.org/java=
/gnu/regexp/>.=20
It apparently doesn't full support Perl5 syntax, but comes close =
enough=20
for most purposes.
> 3) Replacing the =
rather=20
byzantine byaccj process with a grammar etc. built around ANTLR. =
Using=20
the tree support in ANTLR might lead us >directly=20
to:
>
>Parsing the string of characters =
into a=20
parse-tree should be, and I believe is, orthogonal from how/whether =
one=20
generates code from t>hose trees. I would love for someone =
to=20
replace the byaccj parser with an ANTLR-based parser, and I believe =
the=20
result could be slid >in without effecting much else. =
Separately, I=20
have been planning to generate jvm bytecodes from my parse-tree for =
a while=20
now. I'm >perfectly receptive to the notion that ANTLR's=20
tree-grammar stuff might provide a better framework for doing =
so. =20
Again, I don't see the >change as effecting much=20
elsewhere.
Right on all counts, of course; the point =
that I was=20
trying to make--poorly--was that having ANTLR's tree-based =
transformation=20
capability might ease the burden of doing both bytecode and native=20
codegen.
>Btw,=20
a more valuable exercise would be to integrate in ANTLR as a =
quasi-parser=20
generation tool, such that its use to create an E->grammar =
quasi-parser=20
was just one case of that. Were this done, then one should be =
able to=20
turn any existing ANTLR grammar into a >quasi-parser for that =
grammar=20
easily.
Sounds like fun! Is there documentation on =
the=20
quasi-parsing process that I should be aware of?
>4) Bytecode-compiling E rather =
than=20
interpreting. Apart from leveraging ANTLR for this, we might =
also want=20
to look at the open-source GNU Java >bytecode-generation =
library=20
written by Per Bothner at Cygnus as part of his Kawa =
implementation of=20
Scheme in Java.
>
>Worth a =
look. Please=20
gimme a pointer.
<">http://www.gnu.org/software/k=
awa/>.=20
Kawa actually provides a few packages--gnu.bytecode, gnu.expr, and=20
gnu.mapping in particular--that might be of use in creating a =
bytecode=20
compiler for E.
> 5) While we're =
compiling=20
E, let's add support for multiple targets: Java bytecode + ELib =
being=20
the first, of course, and some sort of runtime >(ColdStore =
etc.?)=20
atop EROS being the second.
>The E =
*language*=20
itself could easily be retargetted to a non-jvm platform such as =
Squeak, or=20
perhaps ColdStore. After all, E is an >almost minimal =
lambda=20
language. However, there is the huge matter of the =
libraries. E=20
benefits tremendously by being able to inherit >the Java =
libraries and=20
provide them to the E programmer almost as if they had been written =
for=20
E. Other platforms such as Squeak or >Python also have rich =
libraries it would be good to inherit, but one must choose which of =
these=20
provide the default definition of the >standard E =
libraries. For=20
various reasons, we've chosen Java's =
libraries.
>
>Having made=20
this choice, it would be too much work to build compatible libraries =
on=20
platforms in which the Java libraries were not >available as a =
starting=20
point. Notice that this does *not* pin E into compiling to jvm =
bytecodes. In a system such as Cygnus's, where >jvm code is =
compiled to the runtime formats compatible with corresponding C++ =
code, and=20
where they've already done the work to >reproduce the java =
libraries, we=20
could compile into a native form that used these =
libraries.
Have they in fact reproduced the Java =
libraries? Has=20
anyone tried to build E with, say, GCC 2.95.1's Java tools and =
Cygnus'=20
runtime?
>This=20
was as good as I thought it could get until Tyler made a crucial =
point: A=20
major motivation for E is mobile confined code, especially >for =
Smart=20
Contracting. A confined Smart Contract does not need, and =
typically=20
will not be given access to, many of these Java >libraries. =
The=20
normal environment for a confined Smart Contract will include =
important=20
preexisting libraries, but probably a small >number, and probably =
not=20
inherited from Javasoft's work. Once we've carefully defined =
this=20
environment, we will probably find that we >can easily reproduce =
it on=20
other platforms.
>
>So E isn't easily portable to =
non-java platforms, but the Confined-E subset of E should =
be.
Hmmm.=20
It'd be interesting to see what could happen given, e.g. the =
CLASSPATH=20
project (cleanroom Java libraries) and Cygnus' Java-to-native=20
compilation.
>I'd love to see E running on EROS, but this =
should=20
simply follow from a port of the underlying implementation platform =
(such as=20
a jvm) to >EROS. Once E is on EROS, EROS's capabilities can =
be made=20
to show up in E as E capabilities. I fantasize about E being =
EROS's=20
>shell language.
OK. So who's doing Java for =
EROS?
Anyway,=20
despite having the sense that we're, to one degree or another, at=20
philosophical odds as to what E *should* be, I greatly appreciate =
the=20
information. It gives me a much stronger sense of why E is what it =
is and=20
what's going to have to happen in order to accomplish some of its=20
goals.
Cheers,
--MarkM
Regards,
Paul