[e-lang] [ANN] E-on-Common-Lisp now available

[Kevin Reid]

Warning: The following text contains some unjustified grumpiness.

On May 14, 2005, at 12:53, Mark Miller wrote:

> Hi Kevin, I've started reading the lisp code in your E-on-CommonLisp
> implementation.

Nitpick: It's "Common Lisp" or "CL", not "CommonLisp".

> Wow! I don't yet see the forest for the trees, but I like many
> of the trees. I'm especially impressed by how much of the E libraries 
> are now
> implemented in E rather than primitively. This is great progress!
>
> However, this still leaves, when I printed it, 133 pages of lisp code. 
> If this
> remains the size of the platform specific part that needs to be ported 
> to port
> E, that would be unfortunate. I don't yet have a sense of how much of 
> this is
> essential. How much could easily be moved to E?

Most of it could easily be written in E *if* there were a preexisting E 
system to run it on. E-on-CL is specifically written to avoid becoming 
a "self-hosting" system which cannot be built without an existing 
implementation of its language.

Beyond this, I have made no attempt to *minimize* the Lisp portion of 
the system, but rather (as far as I've thought about it at all) to 
implement each component in whatever language it is most 
straightforward to implement in.

There is also a lot of code which was written before I had a 
sufficiently functional E language implementation, which certainly 
should be reexamined.

If you have any *specific* section in mind, please ask about it. This 
is a huge project, and I don't really want to just randomly select 
sections for rewriting and see if it works.

> How much is testing harness or other infrastructure (e.g. updoc.lisp)?
>
> After these are hypothetically removed, what fraction remains 
> primitive?

The Updoc implementation is entirely contained in updoc.lisp, except 
for some incidentals to loading and invocation in cl-e.asd and 
rune.lisp.

I don't understand what "other infrastructure" is.

> Are there changes we should make to the E language spec so that more 
> of this remainder could be coded in E (like the collection classes)?

I don't know, and I'm not feeling up to thinking about this right now. 
Remind me later.

> Altogether, there are 47 occurrences of "#+sbcl" in your sources. 
> Perhaps
> these should eventually be pulled out into a separate
> CommonLisp-implementation-differences-abstraction-layer, in order to 
> at least
> ease porting E-on-CommonLisp among CommonLisp implementations?

1. util.lisp is partly intended to be that layer.

2. Adding such a layer would require defining general Lispy interfaces 
for the features, and so would add more complexity that is not 
particularly useful *for implementing E-on-CL*. The def-vtable for weak 
references is a good example.

3. Such layers already exist 
(<http://www.cliki.net/compatibility%20layers>), and I haven't 
investigated them. I don't want to put any more effort than I have to 
into making my own until I know what doesn't exist already.

> In elang-nodes.lisp:
>> (def-node-maker |LiteralExpr|
>>   (nil)
>>   ; XXX require value be DeepPassByCopy? be string/int/char/float64?
>>   ((value t)))
>
> The current E language spec does indeed require these to be one of 
> String,
> int, char, or float64. We should discuss whether we should relax that, 
> but if
> we do, I believe it is essential that we require the value to be
> DeepPassByCopy. Without this requirement, doesn't this open a hole in 
> the
> assurance given by your DeepFrozen auditor?

The auditor must, of course, be written to match the language 
semantics. The current auditor would have a hole if this restriction 
were removed, but could of course be written to scan all LiteralExprs. 
(Except that, in the obvious implementation, the auditor then gets 
access to the actual object, which I understand is not desired.)

This could be done either way, and I think it should be reexamined.

> In elib-values.lisp:
>> (def-vtable vector
>>   ...
>>   (:|indexOf/1| (vector subseq)
>>     ; note that this is identical to startOf/1. XXX what to do? is one
>>     ; deprecated?
>
> indexOf/1 and indexOf/2 should be deprecated, and will be as of the 
> next E
> release. Feel free not to implement these, or any other currently 
> deprecated
> methods, in E-on-CommonLisp. Dean, same goes for E-on-Squeak.

I have added notes about this. However, most of the methods of this 
sort were added as needed because code used from the test scripts 
called them.

> In elib-values:
>> (defvar +the-make-weak-ref+ ...
>>   (:|run/2| (referent reactor)
>>     "Make a weak reference to the given ref. If 'reactor' is not null,
>>      invoke its run/0 method when the referent is GCed."
>>     ; XXX what happens for fixnums, null, etc? should we disallow all
>>     ; selfless objects?
>
> I think the answer is yes -- I think it only makes sense to have weak
> references to selfish objects. But I'm not sure; we should discuss 
> this.

I think it depends on the kind of usage, and for some cases it makes 
sense to have weak references to selfless objects: for example, a 
memoizing cache might like to have a weak key map with keys like 
94879235673890167528934765389046784935067453986.

> In elib.lisp:
>> ; XXX thread-safety: make these all vat-local or remove super arg
>> (defvar +the-any-guard+    (make-instance ...
>> (defvar +the-nullok-guard+ (make-instance ...
>> (defvar +the-exception-guard+ (make-instance ...
>
> What does CommonLisp do, if anything, regarding concurrency?

Per the standard, nothing.

The typical implementation might offer some variety of threads, and 
special (dynamic) variables will have thread-local bindings.

> In knot.lisp:
>> (def-fqn scope "org.erights.e.elang.scope.Scope") ; XXX should we 
>> have a
>>                                                   ; non-elang fqn?
>
> Could you explain the question?

Scopes are not tied to the E language itself, and as such perhaps 
should not use elang.* FQNs. This decision has no significance in the 
correctness of the implementation.

> In tables2.lisp:
>>   ; simpler but not-like-Java-E butNot. XXX discuss whether it would 
>> be   : acceptable to use this
>>   ;(:|butNot/1| (map mask)
>>   ;  (e-coercef mask +the-any-map-guard+)
>>   ;  (e. +the-make-const-map+ |fromIteratable|
>>   ;    (e-lambda (:|iterate| (f)
>>   ;      (e. map |iterate| (e-lambda (:|run| (key value)
>>   ;        (unless (e-is-true (e. mask |maps| key))
>>   ;          (e. f |run| key value))
>>   ;        nil)))
>>   ;      nil))
>>   ;    +e-true+))
>
> What's the difference between this and the non-commented-out version?

In this version, the map entries remain in the original order. In the 
Java implementation, the map entries are reordered by FlexMap deletion 
rules.

   ? def m := [1 => "one", 2 => "two", 3 => "three", 4 => "four"]
   # value: [1 => "one", 2 => "two", 3 => "three", 4 => "four"]

   ? m.butNot([2 => null])
   # value: [1 => "one", 4 => "four", 3 => "three"]

The same situation applies to at least without/1 and and/1. I would 
much prefer the simpler versions, but this *is* a significant change.

> To distinguish questions from other outstanding issues, how about 
> "XXX?"?

I'll consider it. Using one tag for everything is clearly nonoptimal.

> If some of the embedded questions are especially pressing, please 
> bring them
> to our attention by email as well. Thanks.

I will, of course.

Incidentally, here's my current set of messages for which I am waiting 
for a reply:

   http://www.eros-os.org/pipermail/e-lang/2005-January/010440.html
   http://www.eros-os.org/pipermail/e-lang/2005-January/010453.html
   http://www.eros-os.org/pipermail/e-lang/2005-March/010515.html
   http://www.eros-os.org/pipermail/e-lang/2005-March/010516.html

> What does define-modify-macro do, as in
> In base.lisp:
>> (define-modify-macro e-coercef (result-type &optional ejector)
>>    e-coerce))
>
> I can tell I need to understand e-coercef before I'll really understand
> anything else.

Have you tried using the Common Lisp HyperSpec? I find it quite 
readable, and surely it would be more efficient and thorough than 
asking me to explain each item.

http://www.lispworks.com/documentation/HyperSpec/Front/index.htm
http://www.lispworks.com/reference/HyperSpec/Front/X_AllSym.htm
http://www.lispworks.com/documentation/HyperSpec/Body/m_defi_2.htm

(If you're using a URL-template tool like Firefox's shortcuts or 
Quicksilver, you can lookup symbols, section numbers, format controls, 
and reader macros via:
   http://www.xach.com/clhs?q=... )

If I've misestimated the difficulty of doing so, feel free to continue 
asking; this just *appears* horribly inefficient to me.

>>>>> I haven't looked at Common Lisp in a very long time. Should I 
>>>>> understand from the above that, unlike Java, the Common Lisp spec 
>>>>> never specified a standard set of command line arguments?
>>>> Yes. The Common Lisp standard says nothing about command lines.
>>> Bletch. Ok, let's do better with rune/clrune.
>>> Does Common Lisp specify a standard native interface (or foreign 
>>> function interface) for linking in C libraries?
>> No.
>> <http://www.cliki.net/FFI>
>> <http://www.cliki.net/UFFI>
>>> Does SBCL?
>> <http://www.sbcl.org/manual/Foreign-Function-Interface.html>
>
> Both seem like more stuff to be added to the CommonLisp-differences-
> abstraction-layer.

That's what UFFI is.

> We should also urge the CommonLisp community to standardize such 
> things.

Um.

You do realize that wishing for such things to be standardized is not 
new to "the Common Lisp community"?

>>>> Many of the test differences, however, reflect behaviors which I 
>>>> would like to see adopted in standard E (or an E standard).
>>> Yes; I am eager for this as well! Let's start with the changes to 
>>> Kernel-E itself.
>> I cannot significantly change Kernel-E without also altering the 
>> parser and expander, which I do not implement. The only change I've 
>> made is that throw() throws SealedBoxes.
>
> What should I read in order to understand the semantics and 
> implementation of
> this change?

What I can think of at the moment:

   * My previous e-lang posts.

   * The "condition rules" section in base.lisp, and places where the 
functions defined there are used.

-- 
Kevin Reid                            <http://homepage.mac.com/kpreid/>