Guest post by Earlier this week, coder and game designer Jonathan Blow gave a presentation on his Twitch channel outlining his thoughts on why and how programmers might go about building a new programming language specifically for game development.
[…]
“We are literally killing ourselves every project, deathmarching to get games done,” said Blow, during a Q&A segment. “It just really doesn’t have to be anywhere near that bad — at least for programmers.”
Does the language really make that much difference to a project “death march”, compared to the epidemic of poor project management? It feels like this drum has been beaten countless times in the non-gaming development world “new language or IDE XYZ with RAD is going to change things!”. And still the real project serial killers remain active: feature creep, underestimating the work required, and frankly in some instances, people just not having their foot on the gas at the beginning of the project while it looks like they still have years of time.
Have you listened to the video/talk ?
(I know it is long, but the first part explains the why)
Edited 2014-09-22 22:04 UTC
I listened to the first thirty minutes.
My issue isn’t that the language or tools used can’t be improved, I don’t doubt him when he says that he could get a 20% or higher increase in efficiency, I just disagree with the initial assertion that it would mitigate project “death marches”. Most of the problems with projects I’ve seen with insane crunches have come around due to poor project planning and would not have been saved by a relatively small increase in tool efficiency, and I think that most of the literature on project failure backs that up. We’ve had significantly improved tools compared to 40 years ago, but the kinds of project failures, death marches and crunch times haven’t changed. If you gain 20% efficiency but then don’t adequately control your project process you will find your budget cut by 20% or scope creep gobble up that 20% anyway.
Maybe if they are more efficient and reduce the time they have to spend on programming by 20%. They will reduce the time spend on the death marches also by 20% ?
Obviously that doesn’t really work, they’ll just end up doing more projects. So there would be more death marches. But they would be shorter, that could be a good thing.
Over the years I have heard the following claims:
– C++ makes you 10x more productive than C
– Java makes you 10x more productive than C++
– Ruby makes you 10x more productive than Java
etc
I did see a productivity boost when I went from C/C++ to Java, because I no longer had to keep track of unbounded arrays, and pointers. I have trouble accepting even a 20% is a claim.
That being said, tools help. So do better libraries. Personally I wouldn’t be so quick to dismiss garbage collection. Were it me I would look at using JetBrains’ Meta (http://www.jetbrains.com/mps/) to make a domain specific language.
But my prediction, he could create the perfect language, but the deathmarches won’t stop.
In my experience deathmarches come from poor planning, bad management, and/or an inability to adapt to evolving situations. A language isn’t going to solve those.
I would.
Anyone who has played Minecraft (and who hasn’t?) knows that unless you give the VM about a gigabyte it gets janky which is really annoying.
And that magic number of megabytes turns out to be about the amount of RAM that the game requires in order to never run a serious GC cycle. So you basically just turned GC off.
Low latency GC uses more and more total CPU time the lower the latency guarantee you want, and at some point you’d be running faster if you were just using reference counted values for everything.
There are some low latency real time GC systems which use a bounded time interval and run only when explicitly called, but these have the problem of getting into situations where they can never complete and just use more and more RAM.
It’s my opinion that real programmers know the memory they’ve allocated and have systems for tracking it and clearing it out when they’re done with it.
I partially agree with you, but in the case of game-world data I think there are some benefits to having a local precise GC. For most of the engine you don’t want GC, but for the big objects that represents the game world it is tedious to avoid cycles.
Add to this that you keep evolving the game design up to release and you can see the value in having more solid memory management than ref counting.
However, you can design the system in such a way that the scan is minimal by keeping the pointers to GC objects on the same cache lines, doing semantic analysis to avoid trees that cannot point to GC objects etc.
Well, no, surely. If you disabled GC, your memory usage would creep up indefinitely. If you ran without GC, the amount of RAM necessary would depend on how long you’re planning on playing.
Only assuming that you don’t pay the price back in locking on reference increments/decrements, and that malloc is likely to be slower than allocation in a decent GC’ed environment (where it’s equivalent to a stack push).
Let’s not make a religious argument out of this. There’s plenty of ‘real work’ done in garbage-collected languages. There’s an interesting crossover point in the D language.
I agree that, for high-performance interactive applications, GC doesn’t seem to be there yet. There’s no ‘killer app’ example that I know of. C# on the 360, for instance, had such awful GC that programmers ended up writing unholy ref-counting hacks – at least C++ has pretty good support for ref-counting. (Some people complain that its support isn’t integrated into the language, but I think std::shared_ptr is pretty damn neat.)
Oh, and, mandatory reference: let’s not forget about region-based memory management.
It’s 2 hours. Of course nobody is going to watch it
.
(Some kind of summary would be useful)
I did. Right up to the Q&A.
You know what it is ?:
Language design is trying to find the sweetspot between (at least):
– overhead for the machine
– mental overhead for the human
– productivity
– how general purpose it is (maybe it’s a Domain Specific Language)
This obviously ends up with very different solutions for different situations.
As someone who is writing their own Z80 assembler with a completely new style of syntax in this field: yes. The technically obsessed cannot always see what a huge difference it makes to be able to express oneself in a language that lends itself to the task. It is about aligning one’s method of thinking to the execution, instead of battling against it.
It’s been hard work even explaining to others why a new syntax is needed, let alone why a micro-comment is necessary when you “already have comments”. Changing your thinking about how you program, and the benefits there of, is not something I think can always be communicated until it’s done and demonstrable.
It’s totally unfair to leave us hanging like that.
Here’s the syntax (note: incomplete, obviously)
https://github.com/Kroc/OZ80MANDIAS/blob/master/SYNTAX.txt
Here’s a sample document:
https://github.com/Kroc/OZ80MANDIAS/blob/master/Sonic1-sms-oz80/main…
Interesting. Observation, not necessarily criticism: That’s not really an assembler per se. That’s a very low level programming language. Something almost at the bare op level, but with enough extensions to make it a higher level that pure assembler. You’re defining specific memory structures and conventions. Whilst that is a very cool and brilliant idea (not original though), you are dictating those structures to any one that uses your extensions. The beauty (and also pain) of assembler, is that those structures don’t exist. With something as memory constrained as the Z80, your extensions might be an issue if used heavily.
How are you implementing this? I would have thought, creating a parser and code-generator that outputs pure Z80 from your extensions would be the best and most maintainable way, then run that through an existing assembler. That way, you can leverage something like the Z88dk’s assembler and target multiple platforms easily. Sorry, I didn’t look at the actual code.
I can’t say I’m experienced with assemblers, I taught myself Z80 for the purpose of disassembling Sonic 1 and writing an editor. I tried a few assemblers out (AS / WLA DX) before deciding to underake my own for very many a reason covered in the video.
I’m pretty sure it’s an assembler though, it doesn’t generate it’s own Z80 code at all. Everything specified in the syntax can be considered macros / pre-processing of your Z80 code. The RAM feature is a special kind of enum that allows for the size of structures so that you don’t have to manually assign every RAM address throughout the codebase — incredibly, other assemblers work exactly like this; just like in the video, the developers didn’t want to treat RAM as its own thing like it really is.
The assembler is written in VB6! It’s fun to code with and not a chore. It’s also not slow — my code can parse 100’000 words in 0.03s.
I’m only going by the docs you linked to, obviously, but you seem to have defined some higher level structures that could be written in pure assembler, but as you say are akin to a “macro”. You do have to wonder where assembler stops and compiler starts though. The features you’ve added are more like the higher level features you’d find in a programming language.
VB6? Cool.. Is the code there too? If so, I’ll take a look and see how easy it’ll be to port it to VB.Net, then port to C#.
The code is at the GitHib repository: https://GitHub.com/Kroc/OZ80MANDIAS
It’s not fully functioning yet, I aim to have it assembling Sonic 1 by the end of the year.
It converted, though not cleanly. Most of the issues are fairly common VB-isms. If did the usual finding lots of unused symbol and stuff like that. The biggest sticking points will be the hacks you have used to get around the fact that VB6 isn’t really fit for the task. If you have access to VB.Net, I’d say make the jump. The language is much cleaner and you have a much richer class library to draw from.
.NET is not the right tool for the job. Having fixed enough computers, I do not wish the .NET runtime on any end-user. You also lose a lot of portability (I work on different computers every day).
Should I have written it in C/C++? Certainly, if I were being paid for it, but this is just a hobby and the assembler is only one part of a larger project and I want to get it working quickly so that I can move onwards and upwards.
If you were really concerned, you would have already created schemas, templates, and the rest.
The logic must be properly expressed as logic, but the 3d equivalent of “sprites” don’t need that, but need a schema so that they can do what you can tell them to do.
I think that it is interesting, but if you haven’t created the schemas or other intermediate things which would be specified in a new language, a new language won’t help.
There will NEVER be a “language for games” because games is too generic a term. They can be literally ANYTHING. There’s no way to make a language that targets ANYTHING and yet can still be considered “for games”. He’s yet another person calling for a language that favors his own vision of gaming in an effort to cut his task load when programming. In other words, you might consider him lazy and conceited.
I’m halfway through the video ATM, and he’s not actually calling for a language “for games”, he demonstrating that games are an example of super-complex systems that are developed in such a way that cause particular grief with how current languages are designed abstractly for “purism”.
Pretty much every language design initiative comes from:
how can we (specific group of people*) be more productive
I’m not a fan of C++. There are parts of C++ which can be productivity killers.
I program in different languages and for example pretty much none of my code uses exceptions even if the language provides it. It doesn’t seem to make the code more readable.
Also not a big fan of a lot of classes to be honest.
With both combined usually my code ends up to be less code because of it. I believe it to be more readable.
The code might be slightly harder to re-factor though. There is always a trade off.
* sometimes this group might just be 1 person or for some general purpose languages: trying to please everyone.
History says otherwise….
AMOS
http://www.ultimateamiga.co.uk/index.php/page,16.html
Lobster
http://strlen.com/lobster
Bliz Basic
http://www.blitzbasic.com/
Professional Adventure Writer
http://en.wikipedia.org/wiki/Professional_Adventure_Writer
Graphic Adventure Creator
http://en.wikipedia.org/wiki/Graphic_Adventure_Creator
That’s his talk. And he’s right. And it’s a general purpose programming language, not just for games designers.
The rest is some combination of shock, denial and bargaining. This is C++ PTSD in action.
No, that’s not what he said at all. He said he didn’t know what the future held for Rust, but as it stood, it was heading in the wrong direction. It would need a pretty hefty about turn to fit his requirements.
I disagree with the author. I see Rust heading in the right direction.
Rust should be a good candidate.
I thought Lua was a popular language used in games. What does he say about that?
Lua is even more a scripting language than Python. (eg. Last I checked, it took a C-style “roll your own as needed” approach to object-orientedness and its type system was primitive enough that all numbers were represented as floats)
While Lua and Python are both popular as “logic-control languages” in the vein of UnrealScript, you still need C++ to actually write the engine and any performance-critical extensions to it.
I think a language based on rule programming would be better. Oh, rejoice, it happened to be one such language derived from LISP that also compiled natively :
http://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
http://c2.com/cgi/wiki?LispInJakAndDaxter
http://all-things-andy-gavin.com/2011/03/12/making-crash-bandicoot-…
Live with it…
Kochise
What a surprise. I just read about that in a textbook yesterday. Apparently it died off because they had trouble finding new hires who knew LISP and it was difficult to share effort with other, more traditional codebases.
Still, fascinating nonetheless. I’m definitely going to sit down and read all of those links when I can spare a moment.
Part answer why LISP not got the attention it deserves as being the most versatile and powerful programming language :
http://www.codeproject.com/Articles/503542/TheplusMythplusofplusthe…
Dude, nowadays I have to code in… Labview to make a living ! How can programmers have invented such a wired mess of spaghetti ?
Kochise
This is the sort of presentation you make when you reach the point in which you blame the language for your inability to estimate things and inability to create projects that are robust enough.
I doubt it is C++, but the way they are using it, and the fact that they oversell projects. I found his presentation questionable.
This is the sort of comment you make when you want to blame programmers for languages that make it difficult to estimate things and/or create robust projects.
If you actually watched the video, you’d know that Jonathan Blow isn’t doing some mystical hand waving with nothing to back it up, but does provide several clear cases where C++ doesn’t meet his needs and also suggests alternatives that meet his needs in a simpler/cleaner way.
– Brendan
I’ve done my share of C++ projects and I never had the difficulty of estimating the task due to language issues. After you set up your project, the language cost tends to diminish in time, becoming minimal to say the least.
I watched the video, and his judgment is poor to say the least. There are so many red-flags from a management perspective in what he says there, that, indeed, his video is interesting to watch and learn from, as a “how not to think about stuff and do things”. I will probably comment his video in depth on my technical blog, but to comment on two hours of mistakes over mistakes will take some time.
… you know, to stop doing death marches?
Even if his technological solution will make games programming 20% more productive – which I highly doubt – it will not solve the problem, it will only make death marches a bit shorter.
Yup, I think that the problem is a management issue more than a language issue. I am awed by how people don’t see this.
If you hate your work and the product is late, then it’s a death march.
If you love your work and the product is late, then it’s like winning a free game of pinball – it’s not a death march, it’s super happy bonus time!
– Brendan
Yeah, that’s why the games industry gets away with mandatory crunch time for months
If only it worked like that…
People can enjoy hiking, but hate being forced to march for days on end.
People can enjoy swimming, but hate being dumped miles from shore and having to swim back for their lives.
The worst part about being on too many death marches or anything similar on the burnout path, is it can take something that you loved to do and, over a few years, ruin it.