A Lexical Mistake

I realize that A Lexical Mistake has been somewhat dark for the last few months so I figured since my current project is starting to look more likely to actually happen I’d put this post up.

See I’ve also been reflecting on the last few projects I attempted and the core problems faced while working on those projects (why I dropped them).

Most of my effort in my personal projects has been focussed over the last couple of years on learning and understanding new programming languages. The problem is that while learning new languages can be very enlightening and quite fun, I found myself wanting to return to my original interest in software development—that is, game development.

The problem is that, while looking at various project ideas, and even attempting a couple (one of which can be found in an earlier post, although the code has been pulled since), I’m still faced with a number of inefficiencies that scream right towards my language design interests.

For a start, if you wish to provide a sandboxed game code environment then the only real option is to combine two languages using the bridge between them as the sandbox. This isn’t really a problem if your interested in simply creating a game now of course, and you don’t have to use a sandbox unless you want to run downloaded game code, but I don’t feel that these technologies really respect the time restricted resources of a small-team game developers (hobbyists, Open Source, relevant Independent gamedev companies).

Respectively, this is also what killed my Haskell gamedev project. That doesn’t mean I don’t intend to look into Haskell further—just not for gamedev tasks in the near future.

And furthermore, both conventional languages and embedded languages almost always require a restart before new code can be tested. I think that by reducing the turnaround time on new game code (or even engine code), the game programming process could be made significantly more accessible to smaller development teams for both smaller ideas or bigger, possibly open source projects.

So what have I actually been doing then?

I’ve actually been putting together a collection of notes and though experiments designed to push my understanding of Smalltalk and programming language design in general enough to put together a cohesive conceptual design for a decendant of Smalltalk, not to mention brushing up on my understanding of existing Smalltalks and the what the various Smalltalk related communities are working on.

Until recently however, I wasn’t convinced that there was an existing language that suited my somewhat unusual requirements for implementing this Smalltalk (or should that be preferences?) Recently I finally figured it out.

As it turns out, I believe that the best language for the early prototypes is The Java Programming Language!

I realize of course that Java won’t give me the best performance, nor will it be the simplest implementation, but when it comes right down to it, I’m just not familiar enough with another language with the necessary UI capabilities that actually conform to the conventions of the underlying operating system. I should even be able to take advantage of several library bindings before tackling C!

So there you have it, a post almost devoid of any real content, announcing my new programming language and both the interpreter and the design in general are coming along nicely so far, so you can look forward to more interesting details in the near future (like the core principles of the language design or the purpose of the language).

Oh, and one thing before I forget, I’ve named it The Mention Programming Language.

After reading Torsten’s nugget of truth in Planet Smalltalk recently I couldn’t help but be reminded of a Smalltalk based language I’ve been throwing around lately. Not because my language is interesting yet—I don’t even have a prototype concrete syntax—I’m actually referring to my immediate reaction when he wrote:

“Smalltalk is not a language — it’s a dynamic object system with the language built in… [sic]

This is, in my opinion, more cautious than it needed to be. Smalltalk really is just a dynamic object system, but beyond that, I would argue that the language is really just a collection of DSLs.

The first, method definitions is kind of the core DSL of Smalltalk. While not necessarily as small as a simple DSL it provides all of the behavioral elements that make Smalltalk code actually do stuff, all of which can be replicated by generating AST objects, programatically.

More obvious however, is the class definition syntax. This quasi-message-send actually causes so many hacks and side effects that suggesting that it resembles any kind of message programmers would be comfortable sending normally is just outrageous! Really it describes the structure of a class and allows us to manage that structure. Oh, we can also file-out that sucker, too!

(The OMeta parser generator can also be used to write methods. The tool support was iffy last time I looked at it though.)

My point is, Smalltalk is really just a collection of DSLs that allow us to effectively program with a pure object system, and it’s this phenomenon that I believe will allow us to dramatically improve both the appeal of the language as well as its usefulness in the future.

— Lorenz

This project is no longer in development.

Of all the programming languages I’m proficient in, not even Haskell is so curiously fun and exciting (in my mind) as the Smalltalk programming language. See the idea behind Smalltalk is deceptively simple, you have objects, which you send messages to. Those messages are handled by methods and finally, both messages and objects may maintain and carry references to other objects. This may just sound like any other object-oriented programming language, but all of this happens at run-time, including the creation of those methods, objects and classes!

Given my affinity for computer games, it’s not surprising then, that I’ve been eyeing off the usage of some dialect of Smalltalk for programming computer games. Even the performance of Smalltalk is competitive compared to many other high level programming languages. Since most high level game code is easier to program with the increased flexibility scripting/dynamic languages provide Smalltalk has always seemed to me, to be a very attractive counterpart to a C/C++ engine.

The same principle could also be applied to the Lua programming language, which is extraordinarily popular in game development right now, which is more or less designed to be attached to an engine written in the standard engine languages, C and C++.

As I pointed out in my last post, originally I only intended to create a game engine in Haskell, which could then help when developing games or frameworks dealing with the high level game code. But to make this easier I decided to put the conventional game development wisdom to good use, and add to the mix a scripting language instead. Also relevant are Haskell’s concurrency capabilities which, I believe, are drawing the most attention from more mainstream game developers.

A Counterpart To Haskell

It might be interesting to note that, in the early design of Fire, the language wasn’t actually going to be based on Smalltalk or any language specifically, rather, I was working out some of the design points to help in taking advantage of Haskell’s concurrency and functional programming capabilities. Eventually I came up with a basic features list along the lines of object-oriented, semi-functional, highly concurrent and prototype based (class-less).

While collecting my thoughts and trying to understand which kinds of syntax would be appropriate I noticed that regardless of which syntax I used, the language would have to almost completely separate the concept of mutable state from the objects themselves, allowing only the use of transaction variables. While I could technically have gone with almost any syntax from there, it’s a pretty sure thing that regardless of the syntax used, Fire was going to be a somewhat unfamiliar thing to work with. This is where Smalltalk syntax lends itself, not being difficult to adapt and extend, while also known for being easy to learn and unusually expressive.

There were some issues to deal with however. To start with, Smalltalk syntax isn’t technically a full language syntax, but only provides the syntax for method definitions. By proxy, the construction of objects and message sends is achieved, with a fair bit of help from the development environment (UI). It also helps to point out that image based languages are notoriously difficult to bootstrap, while many game developers these days use text based VCS anyway, it was obvious I would first have to invent the missing syntax.

After (admittedly) more than a few attempts, and the larger part of a notebook full of musings and hypothetical example code, I finally had a complete syntax and at least most of the basic semantics in mind. The resulting syntax now includes a module level notation, an object literal notation, there is a simplified notation for creating lists and everything else is a consequence of sending messages, as expected.

Difficulties

There are some issues in combining any object-oriented language with functional programming or concurrency. To make things simpler, and because Fire is designed for game development anyway, I’ve kept the basic semantics imperative, or more succinctly the language still has the conventional IO semantics of Smalltalk. The main differences, at this point, are the lack of mutable variables within the language itself (beyond TVars) and the extensive use of threads (asynchronous messages).

To make working with transaction variables easier, Fire does provides properties, implemented as objects responding to a ‘get’/’set:’ property protocol, and a property method syntax which maps a method pair (’var’/'var:’) to another object implementing the property protocol. Any object could theoretically be used here including IORef based objects, but that would generally risk concurrency issues and would probably be a bad idea. This is of course, alongside the usual method definition syntax, and a syntax for creating simply immutable variables, which is syntactic sugar for unary methods.

For those who have worked with Smalltalk, it’s immediately obvious that the main strength of the design has actually been lost however. That it, the main strength of Smalltalk has always been, at some level, the development environment itself. For those not familiar, the built in development tools allow any object, class or method to be reworked at any point while a program is running, even in the debugger. When developing an application, this level of flexibility is essential to the velocity a Smalltalk programmer can work at.

Such a development environment is probably going to be the most difficult thing to replicate in Fire, since both the UI frameworks as well as the tools themselves must be created from scratch. The VM will also likely require a way to allow programmers to ignore the immutability restrictions.

Luckily I do have something along the lines of a plan. What I believe will do the trick here is a simple dual mode VM, allowing (through the reflection layer) the modification of objects, while the finished games will use a release mode, preventing objects from being accidently scrambled. This solves both the development environment concerns, while keeping the language concurrency safe and hopefully simpler to integrate into the Haskell based engine. As as for the UI, I’ll get to that in a later post…

Clearly there’s a lot of stuff going on here, but for now, most of the work will focus on simply getting the spec written and the VM functioning. After then I can start moving only some of the details like the engine itself, and the libraries on either side of the language bridge. Actually, there’s the language bridge too, for that matter.

The purpose of this post is really just to give some idea of the reasoning behind a couple of the high level design choices used in Fire. There are plenty of other topics to consider yet, just at the language level, for example, prototypes systems are even more convention based than Smalltalk since the entire class system is replaced simply by methods for cloning and modifying the definition of objects. In any case, I think it should be interesting to see Smalltalk finally being applied to some serious game development, especially paired up to Haskell as well.

– Lorenz