The Next Mainstream Programming Language: A Game Developer's Perspective :: Redux

[Sean Cavanaugh]

Reposting, as I didn't have my name show up properly due to outlook express 
configuration fun, and any threaded readers are going to camoflauge my 
response to a nearly year old post :)


"Marcio" <mqmnews123 at sglebs.com> wrote in message
news:ecipfc$15v4$1 at digitaldaemon.com...
> http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt


I'm new here but this somewhat old thread starting with this .ppt is very
interesting to me since I work with the UE3 engine as a middleware.  Its
pretty much dead on, as far as what the future is going to be, and that is
in a massively parallel/threaded environment.  Those that do not make the
transition are going to have very noticably 'rigid' game simulations that
are far less capable.  Only the problem from my point of view is there isn't
a single good language make the leap with.  Sure you can have a truely
expert programmer get your threading right with the current tools available,
but the reality is the current tools and languages available to make
threaded programs are pretty bad.  In addition you end up with only 'one
guy' who is responsible for far too much code since he has to wrangle
everyone elses broken stuff into shape, and always on a rather riduclous
timeline.

So how do games, threading, and D all tie into this?

Ok so I've been following D as a lurker off and on.  From the outside as a
C++ programmer, D looks great to me.  I literally get more angry working
with C++ every day.  And its all because doing anything 'cool' in C++,
particularly templates, requires jumping through hoops.  Jumping through
hoops is really the reality of having to deal with language deficiencies.
So looking at D the initial impression is 'sweet' I want to write something
in that.  Except from my world there are several huge problems:

Problem A : Garbage Collection is a dealbreaker.  But not because it exists
or even that is is forced or a default.  We definitely want garbage
collection.  It is a dealbreaker because of how it behaves.  There are
several behaviors that make it such a strong negative as to be a
dealbreaker, primarily the unknown frequency of collections, duration of
collections, and the fact all our threads get suspended.  The more hardcore
games run at a required 60 fps (gran turismo, god of war, etc).  This means
all threads are executing a full cycle in 16.6 ms.  How much time do we want
the GC to spend?  The answer is 0 ms.  Even spending 1ms for any kind of
function in any game is pretty damn slow in a game engine.  If it starts
pushing 5 or 10ms it starts impacting response of input, and noticebly
hitches the rendering, since this hitch generally isn't every single frame.
Consistency of the frame rate matters a lot.   In fact consistency matters
so much that collecting could take 2ms of every 15 and we would be ok with
it as long at was predictable so we can budget the game around it.  Even if
it would only really need 10ms every 5 minutes, that is unacceptable,
because a collector taking 10ms is a dealbreaker.

Problem B : Threading support.  The language of the future addresses
threading head on, understanding that the number of cores on CPU processors
is going to rapidly be in the tripple digits and beyond.  The chip makers
have hit a wall, and the gains are going to come predominantly from core
increases and memory I/O catching back up to the CPUs.  Eventually the line
between CPU's and GPU's will blur quite a bit.  Which means we need to write
threadable code safely, efficiently, without jumping through hoops, and not
even really worrying about it a whole lot.  If our CPU based languages fail
at this, we are going to be ending up writing game-physics raytracing code
on the GPU instead via stuff like NVIDIA's GPGPU.  Which is essentially
going to be stealing GPU performance to make up for the inability to take
advantage of a massively parallel CPU core architecture.  Languages that are
created, or are modified to make this leap cleanly will be the dominant
players of the future.  I believe that if D makes this leap, it will be the
one of them.  Judging by the progress of c++0x, I believe it has lost the
agility necessary to make this transition and will be superceded by another
language at this transition.


My gut feeling says that if the threading issues of are dealt with up front
it would help a lot with garbage collection, since the requirements of
moving the data efficiently in a massively threaded environment would drive
evoloution in the GC system.  Even if the end-result is simply that you can
construct isolated threads to get private GC heaps, and that data must be
marshalled to them as if they were in another process space, it would be an
improvement, because at least then the GC doesnt show down every single
thread, and thin threads can collect very very fast.