Squawks of the Parrot

Choose your (runtime code instantiation) poison

Continuing on from the previous post, let's consider some of the functionality of a configure system. You will, inevitably, want to instantiate some rules at runtime. You don't really need to do this -- a code generator the release manager runs could generate all the code explicitly from templates -- but let's be honest; you're going to want to.

You'll want it for cases where you have a template rule that needs to have a separate instantiation for each file in a directory. You'll have it because when you're designing your rule language you're going to want run a rule for each element in a list or array which means you really need separate instantiations of the rule for each element. You'll have it because it's a shiny thing and programmers just can't resist Teh Shiny. (Admit it, you know that's true)

There are four different ways to handle this.

You could consider each rule as a closure, close over the environment, which includes the current array entry or filename or whatever.

You could have a way to clone an existing rule and make changes at clone time, basically a rule factory.

You could consider embedding the configuration system's compiler in the configuration engine and just treat the code as text, and recompile things on the fly, like perl's string eval.

You could consider a partial compilation system with placeholders, and basically implement Lisp macros to dynamically generate rules on the fly.

Each of the options has its own strengths and weaknesses.

The closure option is the least attractive of the three options. What you're generally closing over -- what the rule depends on, or produces, or the files a template operates on -- is really more metadata than data. And yes, you can close over that, but it feels kind of awkward. Which isn't to say that closing over the environment is a bad thing, even in a functional language with no side effects (which I hadn't mentioned, but there you go), but it's really not the right thing here. The only real upside is that this should work, but since it's hacking an existing concept in a way that's not particularly true to the concept

The rule factory's more interesting. It's a very limited sort of thing, specialized to do one thing and presumably do it reasonably well. That's not bad, if what you want is what it does, and in this case it is. I'm not all that fond of limited solutions like this, though, as they often feel like someone just stopped thinking and made the best of what they had thought about.

The compiler option's a common one, and it definitely has its advantages. The big downside is it means that new rules mean doing text mangling and then compiling the result -- we've got to have the compiler handy, which makes things more complex. Plus this kind of text filtering can be problematic, as we've seen in perl. (Though as we've established, one thing that a configuration processor is going to have to be good at is template processin) On the other hand, having the compiler handy means that you can provide the rule system as text so if something goes wrong the clever end user can fiddle with it to make it right, without penalizing the end user who doesn't want to be clever.

Then there's the lisp macro system, though it's mis-named, or at least deceptive these days since most people think of macros in terms of text, or recorded little action things in a spreadsheet. Basically you manipulate the compiled code, copying, adding and potentially removing chunks of executable as you string it together into a final form. Very much like the rule factory, actually, except without stopping thinking too soon. (The rule factory is, in fact, a subset of the macro system) Creating a new rule is just a matter of copying the code for an old rule and making some judicious substitutions.

Personally, I find I like macro system form best. It doesn't have to be completely exposed in the actual configuration language, merely implemented that way. (Much the same way as compiling Fortran into object code that does all its control flow via CPS -- the fact that the underlying implementation involves continuations is irrelevant (kinda) to the compiled language, which doesn't. It just means you can potentially do Clever Things with less pain)

It also just feels like the right thing to do. If the config system is potentially going to be dynamically generating dozens, or hundreds, of rules on the fly and pitching them at the solver engine, something a bit less haphazard than text substitution and recompilation seems in order.