April 03, 2003

Why not a Lisp (or Scheme) VM for Parrot?

Someone posted a comment asking why, if we're doing closures and continuations in perl, we didn't choose a Lisp VM? (Or, by inference, a Scheme VM)

That's a darned good question. And, given that I'm pretty close to MIT as these things go (it was in comfy walking distance of where I worked for years) I do get asked it a lot.1 This wasn't as common a question for perl 5, and I presume python and ruby, as the language developed slowly and in concert with its interpreter.

The short answer, which made it into the Linux Magazine article (though not yet online--it's in the April 2003 issue, which just hit the newsstands around here) is that I didn't know about the Lisp and Scheme VMs when I started all this. Which is very true, and ultimately the real answer.

A more interesting answer is to the question "If you could do it again, would you choose a Scheme VM?" In this case most likely the Scheme-48 VM, as a number of folks who I respect speak very highly of it.

The answer to that, though, is still No. The reasons for that answer are the interesting ones. If you're a big Lisp or Scheme fan you may want to stop now, as I'm likely going to offend.

Since I'm responsible for making sure Perl 6 runs, I have three big concerns for any solution: functionality, portability, and supportability.

Functionality's the obvious one. If the chosen solution can't run the perl 6 Larry designs, it's no good. The language design is affected to some extent by the back end, and there have been features that have been changed because of features and limitations of the VM, but ultimately if Larry wants it I have to support it and, more to the point, I have to make it fast. That's fine, it's not like Scheme-48 is slow, but the design decisions that drive it are based on the needs of Scheme. Perl's needs are somewhat different--there's that whole "syntax" thing if nothing else. Sure the VM is turing complete, so everything I need for perl is doable, but doable isn't the issue. It's doable quickly. And I'm not sure there's a good match there. Could be wrong, of course, but if it came down to it, changes to Scheme-48 (or any Scheme or Lisp VM) that favored perl at the expense of Scheme or Lisp would be rejected, as they should be.

Portability is a second issue, but it's the one I'm worried the least about. This is, after all, Lisp we're pondering--if a piece of hardware had a two bits, a program counter, and an accumulator someone ported Lisp to it.

Supportability is the third issue. This is the big one, and by far the biggest killer of the deal.

For this to work, I need to support it. And I need to have enough folks in the perl community willing and able to support it. A good working knowledge of Lisp or Scheme is generally absent from the Perl community, and amongst those folks who do have an adequate knowledge of it the general feeling about Lisp is similar to the one I have when I find things with suckers or eyeballs on my pizza. (Which is fair enough, as many folks feel the same way about perl) It's possible that working on a Scheme VM wouldn't require Scheme knowledge, but... I really doubt that.

The development community backing Scheme and Lisp is also much smaller than the one backing Perl. Granted, there's a much higher percentage of people willing and able to bang the metal amongst Lispers than Perl folks, but still... numbers count. I don't need that many people to work on the engine, and I never expect more than five folks competent and active at any one time on the back end, but that core needs to be there, and it's going to have to come from the perl end of things, since I can't count on it from the Lisp end.

So, we have our own VM, one tailored to our needs, which is just fine. There's nothing wrong with another VM in the world--the world is, after all, a large place and there's plenty of room. And it's not like the Lisp or Scheme folks need anyone to choose their VMs for other projects for any sort of external validation.

1 I presume if I was in other places they'd ask the same about one of the Modulas, ML, or Haskell

Posted by Dan at April 3, 2003 01:32 PM | TrackBack (0)

If parrot is going to be optimised for perl6, would ruby, python or, say, C# ever want to compile to parrot? Wouldn't they be better off with their own optimised version of VM, just like perl6 is better off with parrot than with this Scheme-48 VM?

I suppose a point could be made if it enabled the languages to share eachothers libraries, but ultimately the "other" language maintainers will have to support parrot as a compiling target for this to really take off.

Didn't you just kill possible cooperation between these scripting languages by stating that parrot is optimised specifically and explicitly for perl6?

Posted by: Jan-Pieter Cornet at April 17, 2003 08:09 PM

Would python or ruby want to compile to parrot? Sure. The semantics of those languages are subsets of perl 6, and all three languages have many things in common, so an engine good for one will be good for all of them.

C# is less of a good match, but that's OK--it's not one of our primary targets. I'm OK with it if we don't run as fast as a .NET interpreter, though I certainly wouldn't mind being as fast or faster than a dedicated .NET engine.

I presume you're unfamiliar with the background of, and reason for, Parrot, so I can understand the confusion. I should probably explain at some point soon.

Don't forget, ruby and python already have perfectly good interpreters, and while I'm shooting to be better, Parrot's a bonus for them, not a necessity.

Posted by: Dan at April 17, 2003 08:38 PM

Dan writes:
> I have to make it fast. That's fine, it's not
> like Scheme-48 is slow, but the design
> decisions that drive it are based on the needs
> of Scheme. Perl's needs are somewhat different--
> there's that whole "syntax" thing if nothing
> else.

You better stick to the "need Perl hackers to work on the core"-argument. The syntax of the language does not influence the VM.

A *really* good source of information for
implementation of Scheme (and in general:
languages with closures) is the following
bibliography of "Compiler Technology/Implementation Techniques and
Optimization" collected by Jim Bender. There
are links to online research articles.


Jens Axel Søgaard

Posted by: Jens Axel Søgaard at July 18, 2003 01:23 PM

You misunderstood the comment. The syntax issues are an issue for the VM because, being a compile-and-go language, the compiler's integrated into the VM. Scheme's the same way, of course, but perl's syntax is significantly more complex than Scheme's--we couldn't even share the same tokenizer. That means for us to use the Scheme VM we'd have to either rip out and replace the compilation unit or load on top of it. The first means a break from the stock distro of the VM, the second means a slow compilation phase since we'd probably end up having to write the compiler in Scheme. The mismatch between Scheme's synax and Perl's syntax makes riding on top of the Scheme VM more difficult.

Posted by: Dan at July 18, 2003 02:01 PM

> You misunderstood the comment. The syntax
> issues are an issue for the VM because, being a
> compile-and-go language, the compiler's
> integrated into the VM.

In the context where you made the comment
(choosing between VMs to start developing from) I wouldn't consider it an issue.

If you need support for reading special syntax in an existing VM, all you need to do is to add an extra primitive. This is trivial compared to all the other design decisions that needs to be done when designing a VM. What really counts are heavy weights such as function calling protocol, data representation and friends.

And (as a side note) you don't actually need a tokenizer in the VM, you could manage with read-char and peek-char. Note that 'read' is marked as a library procedure in R5RS meaning that it can be implemented in terms of other more primitive operations (and thus the compiler writer could choose to implement 'read' in Scheme and not in the implementation language.


The same will be the case for Perl.

Posted by: Jens Axel Søgaard at July 26, 2003 07:03 AM