A Lexical Mistake

After reading Torsten’s nugget of truth in Planet Smalltalk recently I couldn’t help but be reminded of a Smalltalk based language I’ve been throwing around lately. Not because my language is interesting yet—I don’t even have a prototype concrete syntax—I’m actually referring to my immediate reaction when he wrote:

“Smalltalk is not a language — it’s a dynamic object system with the language built in… [sic]

This is, in my opinion, more cautious than it needed to be. Smalltalk really is just a dynamic object system, but beyond that, I would argue that the language is really just a collection of DSLs.

The first, method definitions is kind of the core DSL of Smalltalk. While not necessarily as small as a simple DSL it provides all of the behavioral elements that make Smalltalk code actually do stuff, all of which can be replicated by generating AST objects, programatically.

More obvious however, is the class definition syntax. This quasi-message-send actually causes so many hacks and side effects that suggesting that it resembles any kind of message programmers would be comfortable sending normally is just outrageous! Really it describes the structure of a class and allows us to manage that structure. Oh, we can also file-out that sucker, too!

(The OMeta parser generator can also be used to write methods. The tool support was iffy last time I looked at it though.)

My point is, Smalltalk is really just a collection of DSLs that allow us to effectively program with a pure object system, and it’s this phenomenon that I believe will allow us to dramatically improve both the appeal of the language as well as its usefulness in the future.

— Lorenz

I’ve been mulling over my old game development hobby for a little while now, and after messing around with Python a little I noticed that once again, there seems to be very little in the way of promising concurrent game object level languages floating around. At least languages that I’m aware of.

What I’m talking about exactly is game languages which allow the game objects themselves to participate in concurrent programming practices, without using intricate and complex locking sequences, or losing the right to talk directly to engine/library level services like physics or high-level game state (moving between the level and a menu for example).

Instead, almost all game level languages at the moment seem to be evangelizing the importance of being script level uncompiled beasts that require little to no real programming experience before becoming a productive member of the game code team!?

Of course I’m not saying that this is an inherently bad practice with the current level of technology floating around, but I do have to wonder whether this is still relevant for small team game development in the near multi-core parallelism future. Perhaps too far, but I think also relevant is the much slower pace of Open Source Game Development, which realistically speaking needs technology almost 3 years ahead of its time just to break even on an initial 1.0 release!

My question is actual reasonably simple despite the cynical tone up to this point… Can we realistically design a new object oriented DSL that contains the necessary concurrency techniques to allow programmers to write reliable and correct concurrent game code. This actually goes beyond just the engine code for which we’ve always been able to reasonably easily move to a more sophisticated language like Haskell but more towards the game specific stuff, which we cannot afford to bog down in complicated syntax and general side effect hostility.

I actually believe this is possible, and I’ve already begun planning the first steps towards writing some basic engine code that will underpin the project but I’m definitely more interested in seeing how effectively I can pin down the STM concurrency model into an object oriented syntax.

Ok… what about the language stuff…

I’ve actually been looking for a good excuse to both learn Haskell more fully (I’ve really only coded trivial examples in it so far), as well as properly exercise the Parsec combinator parser and finally, and perhaps most of all, design a new object model and object oriented programming language using that model.

The last, ironically may have to be restrained a little, but a concurrent perhaps slightly more conventional object oriented language may not be so bad. The point is that the project will involve a fairly comprehensive object oriented programming language, and one that is not only fully concurrency enabled but also tailored to game development tasks.

This is probably a good place to point out, that the conventional object oriented language I’m talking about here doesn’t include either C++, Java, Ruby or even the CLOS, but rather I’m talking specifically about the Smalltalk family of languages here. Its the work done on Smalltalk/Squeak, Self, Slate and more recently Newspeak that I find the most interesting, and the dialect I develop will likely incorporate many of the simpler innovations in the object systems they use.

The most critical issue that I’m aware of is the complexity added by the multi-dimensional objects used by most OO languages these days (including Smalltalk in this case). Rather than allowing the object state to exist in two (or more) dimensions I’ll be limiting them to a single flattened dimension similarly to record types, and the programmer will need to use either delegation/cooperation or composition in order to accomplish tasks in a reusable fashion. If you didn’t follow the dimension of state thing I’m actually just talking about Inheritance (another dimension might be Aspects, et cetera).

While I don’t want to go to much further into the syntax I would like to note that my intentions for this, apparently built in, concurrency will probably be in the form of an asynchronous send, paired with transaction support (which will actually be the only way to guarantee that data is written correctly). The asynchronous send, by definition, allows the receive to process it in a different thread… but since my little language is probably going to be more or less interpreted or at least green threaded, I’m pretty sure that I can just replace the async send operation as a plain old spawn operation.

The async message send in this case should provide a convenient way of signaling between game objects, and paired with the transactions (noting that very few Erlang processes don’t actively apply transactional semantics as well), I should have an effective way of implementing fully concurrent game code.

Well then… hopefully I don’t get killed between all of these projects and we’ll see something interesting come out of this stuff. Until then…

– Lorenz

Ok, my prototype is slowly moving along, but one thing still alludes me. What is the best set of objects to bootstrap the system with, and which objects should be written on top of the system after the system is running?

The final version of the language was meant to be based on a concept of method inlining. Basically, since most memory accesses are more or less constant events within a message, and even when they’re not, most of the code could be inlined further down the message stack, we could rather than using conventions or making the language very memory neutral/high level, we can instead allow programmers to directly manipulate their objects.

Of course this doesn’t really help us now (the current prototype is based on Squeak, which lacks the effective memory model to warrant using raw memory layouts yet), but even if we did implement this approach, we would still require a more generalized object layer on top of it. For this reason I believe that implementing a more restricted object model is in order.

To start with, I believe the basic object structure will be needed (Objects must be defined by the external system), and from there I will attempt to describe the general structure of the language using the Multi-Method equivalent of delegation (traits). This will include all of the necessary interfaces required to make the language self sufficient post bootstrap (except the Virtual Machine itself which will be rewritten in C or Haskell until the language can compile itself).

There will also be several objects that need to be constructed right from the word go, which includes, roughly, Integers, Floats, Strings, Symbols, Lists/Arrays, Dictionaries, Classes, Traits, and the base level behavioral interfaces that apply to all objects. These objects will be provided from Squeak directly, or will be implemented on a Generic object Squeak class, which can provide all of the necessary behaviour to bootstrap language level objects.

One last point I’d like to make. The behavioral interfaces do not actually apply to Mention objects specifically, but rather describe how the programming environment reacts to objects in general. Any object may be added to the Mention system simply by defining the fundamental interfaces required (essentially just some relative typing information for the dispatch engine), and then it can be used anywhere in Mention that actual Mention objects can be! In-fact, this goes so far in the language that there will very likely be components that do not even encode type information into objects for the sake of performance (no object headers or garbage collection). This is possible if the interfaces do not require any dispatching at runtime (otherwise the lack of type information interfaces will cause compilation errors!).

Anyway, hopefully I’ll have the prototype more or less working soon and you’ll be able to check it out for yourself, though it won’t do very much at the moment;)

– Lorenz