Here's one that caused much consternation at times. I ruled that the destination of an assignment must always exist. That is, code that looks like:
add Pdest, Pleft, Pright
needs to have a real Pdest -- that is, it's an in parameter, not an out parameter.
Some people hate this. "Why can't my add function create the destination?" they cry.
Well, because in many cases the destination already exists. Creating a new PMC unconditionally means generating a lot of temporary garbage. More temps means more garbage for the garbage collector, more temps needed from the allocator, and more time spent in bookkeeping with these temps.
Consider, if you will, these two scenarios:
a = b + c
and
d = e + f + g
where we're only interested in the "e + f" part.
In the first case, the destination already exists. (We can't just arbitrarily whack destination name/variable bindings, because the destination might be active data. That warrants a What The Heck Is post of its own. Active data has been the bane of many optimization strategies, but it's also phenomenally useful, and more importantly an integral part of Perl so there's no ignoring it) There's no need to create a temp for the "b + c" addition, as we already have a spot to stick the result -- the PMC for a. There's a possibility in the first case that the destination PMC is of an inappropriate type, in which case a temp PMC needs to be created and handed to the destination PMC for ultimate assignment/value snagging.
In the second case, a destination doesn't exist. Since this is known at compile time, though, the compiler can just emit code to create a temp of the appropriate type and we're fine there, or emit a generic Undef if the compiler doesn't know. (Depending on the language it usually will know the appropriate type to emit)
So we've got three possibilities -- destination exists and is OK, destination exists and isn't OK, and destination doesn't exist. In the second and third cases we have to create an intermediate PMC, and in the first case we don't. If, on the other hand, we had the add op unconditionally create PMCs, then the first case creates a temp PMC which is then discarded. More garbage than we need.
Things get more interesting with extended expressions, like:
a = b + c + d + e + f
If add creates a temp, then we have created 4 temp PMCs. (for the b+c, that result + d, that result + e, and that result + f) On the other hand, if we make the destination exist, we need exactly two temporary PMCs. In fact, we need only two temp PMCs no matter how complex the expression is, as long as there are no function or method calls in it. That's a lot less garbage, especially if you're in a loop.
You may be thinking "two temp PMCs? How, exactly?" That's easy. There's a temp needed for the b+c part, and a second for the result + d. The first temp can be reused again to add e in, then the destination to add in f.
"What about continuations" I hear you cry. (Though, I suppose, not necessarily about this specifically) "They could happen anywhere!" No, they can't. They can only happen explicitly, or inside functions or method calls. Parrot specifically forbids continuations to escape from within vtable functions so there's no way our expression there can be resumed in the middle, which means that the compiler's free to reuse its temps. Consider, for example, the following:
foreach a (@foo) {
b = b + c * a
}
Where we're iterating through the @foo array. No function or method calls, which means there's no way that continuations could be taken. If there are a thousand elements in @foo, if we have our math ops create temps, we've created two thousand temps. If we have the destination exist, we have to create... two. Even if we're Really Clever and explicitly free up the created temps, that's still two thousand calls to the allocator and two thousand calls to the free routine, instead of two to each.
So there you go. If the destination has to exist it means that, worst case, performance is no worse than if binary ops create their own PMCs, and in some case significantly (orders of magnitude) better.
I've been seeing the webserver here getting actively referer-spammed (that is, connections come in with obviously fake referring URLs) by a whole cloud of machines out on the 'net, something that I assume is pretty common. The requests come in from IPs that haven't looked at other pages and don't look at other pages (at least not with a valid referrer), and given that they all refer back to a single site I'm working on the assumption that someone's bought a whole wad of compromised machines and is working on spamming log files so that their URL shows up when people run analog and tools like it to up their pagerank.
This, needless to say, pisses me off. It also eats up some of my bandwidth and, while I'm not paying for it, it makes other things on the server sluggish.
So, in the spirit of sysadmin bastardy, I'm black-holing these hosts. They're either compromised machines or open proxies, and either way... pfui to them.
If for some reason you suddenly find there's no route to the webserver here then that's likely why. (Or we've been hit with a power failure that's lasted longer than an hour, but those are pretty rare for me)
Welcome to the shub-internet -- you'll not find a more wretched hive of scum and villainy. Enjoy your stay!
On the top of the list of things I'd do differently in Parrot's design is fully embrace multiple dispatch. I mean fully. Any operation with two or more operands would be multiply dispatched, and this includes assignment.
When Parrot started MMD just wasn't on the table. Oh, sure, there were overloaded operators, but that was all left-side-wins stuff, wedged into the PMC vtable. We introduced a standard MMD system, with the assumption that PMCs which wanted to do it would have their vtable slots use the standard system so there'd be cross-compatibility. That then, after a while, led to the realization that we could shrink PMC vtables a lot, simplify the internals, respect a default left-side-wins anyway,make things faster, and generally make things work nicely if we just went entirely MMD for binary operations with some proper defaulting. Unfortunately this was after we'd tossed the keyed versions of the different binary ops, a loss I still think is a waste. That's a rant for another day, though.
The one binary operation I failed to consider was assignment, and it's an important one, and parrot didn't multiply dispatch it. Big mistake. (One that's fixable with the current parrot design, interestingly enough, the same way that we could make the binary operations use MMD without affecting the bytecode in any way)
Now, the whole point of MMD, at least as far as I'm concerned, is to allow you to cheat like hell when you know it's safe. That is, you provide a nice, generic, safe interface that, while potentially a little slow, lets you have those nice black-box data structures we're always told we really want (and never believe until it bites us hard, usually a year or two too late to fix the problem) while still being as fast as it possibly can be in those cases where we know we don't need to be careful.
For example if we're assigning an Integer to an Integer there's really no need to go jumping through any sorts of hoops -- the Integer class knows what the internal structure of an Integer looks like (we can hope, at least, and worry about the authors of the class if it doesn't) so making function calls to get values is silly. That is, the assignment function for an Integer-> Integer assign should look like:
dest_pmc->cache.intslot = source_pmc->cache.intslot
rather than
dest_pmc->cache.intslot = source_pmc->vtable->get_integer(interpreter, source_pmc)
(modulo wrong off-the-cuff C indirect function call syntax). Of course you can't have the first form as the standard assignment function, since it's wrong for so many things. Indeed, the only thing you can really do is have the standard assignment look like:
dest_pmc->vtable->set_pmc(interp, source_pmc, dest_pmc)
with the set_pmc function for the destination then:
dest_pmc->cache.intslot = source_pmc->vtable->get_integer(interpreter, source_pmc)
Which is definitely sub-optimal in specific cases, while OK in the more generic case. We could, of course, throw a flag test in the assignment function for the Integer class, but we know from experience that flag tests are more expensive than they're generally worth, are an inextensible pain since they lead to if ladders at the top of functions, and are an indication that we should be doing MMD anyway.
There you go. Straight assignment should've been MMD, but it wasn't, as much for hysterical raisins and evolving understanding as anything else, and it could be made multiply dispatched if
That then leads to the question 'should the full binary operation be MMD on the destination?' That's a valid question, since parrot requires (or did require) that the destination for a binary operation exist. There's been much bitching about that on the list, but it's a pretty significant win in terms of temporary objects created (or, rather, not created) and going on about that's a topic for another WWIT anyway, so I'll stop with the explanation there.
So. Should "a = b + c" dispatch on just the types of b and c for the addition, and then on the result type and a for the assignment, or should there be just one big dispatch on the types of a, b , and c?
That's a good question. I dunno what the right answer is. Or, rather, both Yes and No are perfectly fine answers, and the best one is a matter of figuring out which would be the common usage and going with that.
My gut feel is that generally it's not a win, so the two-step dispatch is better. On the other hand, a good case could be made for doing the three-arg dispatch. Pleasantly, since as far as the bytecode is concerned It's All Magic Anyway, either one could be chosen and later on the other could be switched in.
Who knows, if Chip wants it, this could go into parrot now. Certainly's going to in my tree.
This was one of the things I really wanted to get into parrot. Granted, mainly to support playing Zork, but as a side-effect we would've gotten the capability to load in JVM and .NET bytecode, along with python bytecode, and for the really adventurous platform-native executables. (Or even better, executables for other platforms)
What I'm talking about here is giving the bytecode loading system of parrot the capability to have special-purpose bytecode loading libraries which can be loaded at runtime, as well as the capability of detecting what type of bytecode is being loaded and handing it off to the right loader. Parrot does have a version of this built in, but it's relatively rudimentary.
What I wanted to do was have a general-purpose mechanism in place to allow registering a loader and the conditions under which it would fire, and then have parrot walk the list of loaders every time a file was loaded up. This was already sort of necessary, and sort of implemented, to dispatch based on the extension of the file loaded in -- that's how parrot manages to handle bytecode, pasm, and pir files transparently. It's a bit hardcoded, though, and I'd rather it wasn't.
Why allow runtime additions to the bytecode loading system?
My personal favorite, all jokes aside, is the z-code loader. I fully expect that nobody sane will deploy it in a production environment, but I personally think it'd be really cool to be able to do:
parrot lurkinghorror.dat
and find myself on the campus of good old George Underwood University.
That aside, there are a lot of different bytecode engines out there, and there's no real reason not to be able to do a transform from one to another. Combined with the loadable opcode library facilities that were supposed to go into parrot there's no reason parrot shouldn't be able to handle other engine's bytecode -- the simplest way is to have a library of opcode functions that exactly match the functionality of the original bytecode and do a transform from the original bytecode to parrot bytecode, something that'll likely be mostly just an 8->32 bit word transform with a little bit of opnumber munging. This is something that's pretty easy for parrot and much less easy for most other VMs, since we have such a huge range of opcodes. Doing it on the JVM or .NET engine would require a more complex transform of the inbound bytecode. (Which isn't a bad thing, of course. It's just a thing)
More usefully, if you consider source code just an odd and somewhat densely packed bytecode, it means that allowing this means that all you need for parrot to properly dispatch source to a compiler is a bytecode loader that takes the source, compiles it, and then executes it. Want to handle ruby? Have a bytecode loader that dispatches all the .rb files to the ruby compiler and runs them. Tcl? Same thing. Heck, do it for all the languages that have registered compilers. (Though this would argue for a deferred, just-in-time library loader so you don't pay to load in all the language compilers and bytecode translation modules every time parrot's started, but that's not a big deal)
This is one of those things that doesn't really have a place in one of the other parrot sub-categories, so I'll just go on about it here.
How the heck was parrot supposed to do context? Context, here, being "what does my caller expect me to return?" For perl 5 this means either scalar (it expects a single value back) or list (it expects multiple values back) but perl 6 made things a bit more complex. (Which is a separate topic, for someone else to talk about)
This was one of the things that really bugged me, and you'll note the glaring lack of context information in parrot's calling conventions. This was very much on purpose -- Larry'd waffled back and forth some on how much context information should be available to subs, and so I'd put it off for a while. This was one of those things that just extending the perl 5 meaning wasn't enough for.
You'll note it never resurfaced, at least not in the calling conventions.
Why? Because it just wasn't needed.
If you look at how parrot calls subs and returns from subs, you'll of course notice that they're done identically. Same registers filled in, same method of invocation, same everything. That's because the way that parrot uses CPS means that calling subs and returning from subs works identically. Return values are just parameters passed in to the return continuation.
Which means that there's no difference between the expected return values and the expected passed-in values for a sub.
Which means that the return context is just the prototype for the return continuation.
So, if you want to figure out what context you were called in, you just need to fetch out the prototype property from your return continuation and you're set.
No muss, no fuss, no fancy nothing.
Granted, this then raises the question of how to specify a prototype for a sub object, but that's a separate question, and one that needs answering anyway, so the answer to it answers the "how do I specify the return context" question.
I did an interview last month with Jack Woehr for The Perl Journal. Nothing big -- something of a project retrospective and a why I left parrot thing. Well, it's up, if you're interested. (Pay-only, though, and as such I've not seen the final draft, but I assume I don't come across as too much of a blithering idiot)
This isn't the full project retrospective, something I'm working on separately. (It wasn't a bad run, all things considered. There are a few things I'd do differently, and some things that did work out remarkably well. This is all project management and political introspection -- the technical stuff is all separate and it's arguably premature to poke at that until there's an actual release of parrot)
From the very start, we declared that we'd have a mostly-universal bytecode format. That is, assuming you built parrot to use 32 bit integers for opcodes, bytecode you built on one machine would run anywhere. Not necessarily without translation, but parrot would provide that translation automatically.
Why? Simple. Binary distributions and multi-platform shared installs.
Now, before you get your freak on over the lack of source, a traditional perl community hot-button issue, remember that parrot's a multi-language interpreter, which means you might not have the compiler for the language in question. Just because you've got parrot with the perl and python compiler modules doesn't mean you've got the ruby, prolog, C#, BASIC, and Intercal modules installed, so you're kind of out of luck there even if you do have the source. (And the source can be embedded in the bytecode as metadata)
There are also times when binary installs are better, even with complete internal distributions -- you don't have to worry nearly so much that Joe from Accounting will use his "mad programming skillz" to helpfully fix the bugs in the app you're deploying. (You know those bugs -- the ones keeping Joe from trashing the database and destroying everything done since the last backup)
And, of course, combined with a linker, being able to do universal bytecode means you can link your program into one big file with all the bytecode for all the libraries built in and distribute it so people only need a base parrot install and nothing else. (Or you can then run it through the bytecode->executable converter to get a single-file executable) You are, of course, completely responsible for the social implications of that (including the legal bits) but we only do the technical bits, social things are your problem.
The multi-platform shared install is important as well, and how much is something you don't tend to notice until you've had to manage a shared install of some application that's used on multiple operating systems and hardware platforms. (Though this is becoming less common as the various hardware platforms and operating systems die out) That is, you've got a shared app install on some NFS mounted volume somewhere and all the systems on the network use the shared install.
Now, of course for this to work you need to build your main system (which in this case would be parrot) on all the different platforms, which is a pain. It also means that all binaries need to be built on all platforms, which is a really big pain when upgrade time comes if you've got modules that have binaries.
Parrot's NCI system should make modules with a C component less common, but it's still handy to do compilation to bytecode, and universal bytecode means you get to do this once and deploy it portably. This is useful in cases where compilation from source is slow (if, say, you've got a language with a strong optimizer, slow compiler, or one that triggers degenerate behaviour in parrot somewhere) or where the compiler module itself is platform dependent but the output bytecode isn't.
Anyway, the more you can share across platforms without having to do anything at all special, the easier it is to pass things around and make everyone's life easier. That's a good thing, so far as I'm concerned.
I tracked down the bug that caused Devel::Size to segfault when tracing some code refs under perl 5.8.x (where .x is less than .7). One of the optree nodes was being mis-identified somewhere, either by my code or by some of the perl internals, dunno which. Don't care which, I suppose.
Anyway, Devel::Size 0.63 is off to CPAN, and should fix all the known issues that Devel::Size has. Now all that's left to do is finish figuring out how big the regex nodes are (and where they're stored) and final bits digging out format memory usage.