Had dinner wednesday night with a whole bunch of folks in the .NET and dynamic languages community at OSCON here. It was interesting, though I'm sorry I missed the .NET/Dynamic Languages BOF earlier in the day. (I was setting up food for a party and, after all, one must have priorities :) It was a good opportunity, as the static and dynamic language folks tend to talk past each other, and .NET is definitely in the static language camp.
Anyway, one of the things that came up at the dinner was the issue of "Why do you need that feature? What are you doing with it?" It's the standard question, one people ask when you're looking to do something that they either don't understand or do understand but don't see the point of because almost nobody will use it. It's a good question, too, one that should be asked. But...
The question misses the mark in some ways, because people tend to merge the use of a feature in general with writing code that explicitly uses a feature. For example, take Perl's source filter feature. (I bet you were thinking I'd trot out continuations) It lets you install an arbitrary piece of code that can perform any sort of transformation on source code before it gets fed to the parser. It's cool, but fiendishly difficult to use. Even with the Filter::Simple module it's a major pain. And it was added in for the sole purpose of making on-the-fly decompression/dearchiving of source files possible. (So you could keep compressed and/or tarred source)
Now, there are a bunch of modules on CPAN that use source filters. All but one, so far as I know, are gag modules. (Many of the Acme modules, including Acme::Bleach and Acme::Pony, are source filters) Actually using source filters to do real things is massively difficult, so arguably they're of no use, because nobody uses them, right?
Well... Nobody uses them directly. Indirectly, though, they get used by a lot of people, because the SWITCH module uses them, and people use that. SWITCH provides a very nice switch statement to perl, nice enough that it got bundled in with 5.8.0 as part of the standard library. The same thing can be said about Lisp macros (though I don't see what the big deal there is), continuations, coroutines, threads, and even closures. Nobody, to a first approximation, uses them in general, and certainly nobody uses them in languages that don't have them. And they're all hard concepts in many ways. (Yes, even closures--you've just gotten used to them) So shouldn't they be left out?
The answer is "Not because they're hard concepts." Which isn't to say they shouldn't be left out, just not because they're hard. Each of those concepts has a number of very interesting things you can do with them that's between difficult and impossible to do without them. (Yes, yes, I know--with continuations and Lisp macros you can do anything, as long as it still looks like Lisp...) Many, perhaps most, perhaps even the overwhelming majority, of programmers will never use them well. That's fine.
They aren't there for the majority of the programmers using the language.
That's the point people miss. You don't provide source filters, continuations, macros, or whatever because everyone will use them, you provide them so the two or three people who will use them can produce things that would otherwise be insanely difficult. What people do use is what those few folks made, and that is what makes the difficult stuff worth providing.
Posted by Dan at July 14, 2003 03:06 PM | TrackBack (1)that's a fantastic argument.
i'll have to remember that for the next time the subject comes up.
btw, loved your talk on parrot at OSCON. i was the guy asking about deterministic finalization. it's nice to see that so much is working. i lurk on the mailing list, but even so i hadn't realized how much progress had been made. it's easy to get bogged down by the amount of things left to do, and forget how much is right there working today.
Posted by: garrett at July 14, 2003 06:58 PMWell, taking your aside semi-seriously, it is possible to get non-lisp looking (common) lisp code, using Lisp macros. A canonical example is various infix math macros that exist. Admittedly, it's much, much easier to do lisp-looking lisp macros, but... ;-)
Posted by: Brian at July 16, 2003 03:48 PMYeah, I've heard the "It doesn't have to look like Lisp!" argument about macros, but all I've seen is code that looks like:
(print (1 + 2))
instead of
(print (+ 1 2))
I suppose that's a difference, but it's still paren-grouped, space-delimited, functions all over code.
Not that it's a bad thing, but there's more to the feel of Lisp than prefix vs infix for the math operators. Every example I've seen people trot out, and I saw Olin's Lambda, the ultimate little language presentation at LL1 which revolved in part around a bunch of shell-like macros, immediately pings my "It's Lisp!" sensor.
Maybe I just need to get out more. :)
Posted by: Dan at July 16, 2003 04:05 PMCase in point with the 'hard things allow opportunities for good voodoo' argument. I've been thinking about the possibilities for a kick ass debugging/refactoring/development environment for Perl 6. Those possibilities are there because (hopefully) I can extend the various core classes to allow me to hold handy meta data, and I can use the macro facility to, for instance, make every block entrance hang an 'entry' continuation off the current return continuation so that, when I run something under the debugger and it crashes out with a stack trace I can choose whereabouts in the call chain to start running in step by step mode. (And, for bonus points, I can 'bookmark' that entrance continuation in some debugger variable so I can try different things and still get back where I started...)
I really should start writing some proposals for the Perl 6 debugger...
Posted by: Piers Cawley at July 16, 2003 10:28 PMA C# microsoft person once said that C# doesn't have continuations because... he never quite understood what they were good for.
I understand that one, actually. I did the BASIC->assembly->Forth->C route for languages (with a few side trips) so when I got to Perl I didn't see the point of closures, and until relatively recently I didn't see the point of continuations. (Objects and exceptions were also fuzzy things that I sort of saw the point of, but until I started using them I didn't fully grasp it) Until you actually use a tool you can't fully appreciate it, and for tools that do things you're unfamiliar with can't really understand it.
Not understanding a thing is one of the good reasons to leave the thing out, because you don't understand the ramifications which may impact the implementation. (Not understanding continuations or closures in a system with zoned privileges and capabilities is a good way to open massive security holes, for example) I'd rather a feature get left out because the designer/implementer didn't understand it than see it included badly trying to make what they sort of think it should be work.
Posted by: Dan at July 18, 2003 11:34 AM"We put that in there for Damian. He'll write the usefull stuff."
If I had written those words, I would have done it with Damian's face in my mind. Or maybe his solution to a particular problem - the solution which was simplest to the programmer required the use of Quantum::Superpostions. Posted by: Chris Cantrall at July 18, 2003 11:58 PMWell... Nobody uses them directly . Indirectly, though, they get used by a lot of people,
...
Each of those concepts has a number of very interesting things you can do with them that's between difficult and impossible to do without them. ... Many, perhaps most, perhaps even the overwhelming majority, of programmers will never use them well. That's fine.
They aren't there for the majority of the programmers using the language .
That's the point people miss. You don't provide source filters, continuations, macros, or whatever because everyone will use them, you provide them so the two or three people who will use them can produce things that would otherwise be insanely difficult. What people do use is what those few folks made, and that is what makes the difficult stuff worth providing.
nicely put. very nicely put.
now, let me take your aside farther you probably wanted to take it.