D on next-gen consoles and for game development

[Jonathan M Davis]

On Friday, May 24, 2013 09:42:10 Manu wrote:
> On 24 May 2013 09:02, Jonathan M Davis <jmdavisProg at gmx.com> wrote:
> > On Thursday, May 23, 2013 22:02:05 QAston wrote:
> > > I think that Phobos should have some support for manual memory
> > > management. I don't mean clearing out the gc usage there, as it's
> > > fairly obvious. I rather think about something like
> > > unique_ptr/shared_ptr in the std. I think unique_ptr can't be
> > > implemented without rval refs, also C++ sollutions may not fit
> > > here. Anyways, now it's not so straightforward how to live
> > > without gc so standard sollution would be really helpful.
> > 
> > We have std.typecons.RefCounted, which is basically a shared pointer.
> 
> I've always steered away from things like this because it creates a
> double-indirection.
> I have thought of making a similar RefCounted template, but where the
> refCount is stored in a hash table, and the pointer is used to index the
> table.
> This means the refCount doesn't pollute the class/structure being
> ref-counted, or avoids a double-indirection on general access.
> It will be slightly slower to inc/decrement, but that's a controlled
> operation.
> I would use a system like this for probably 80% of resources.

We use smart pointers where I work and it's a godsend for avoiding memory 
problems. We almost never have them whereas the idiots who designed the older 
software used manual refcounting everywhere, and they had tons of memory 
problems. But while we need to be performant, we don't need to be performant 
on quite the level that you do. So, maybe it's more of a problem in your 
environment.

> > Also, it should be visible in C++/D that D can really deal with
> > 
> > > manual memory management conveniently - when I checked out Dlang
> > > first time I felt very disappointed that "delete" operator is
> > > deprecated. "So - they advertise one can code without GC, yet
> > > they seem to deprecate the operator" - false claims discourage
> > > people from using new languages.
> > 
> > delete is only used for GC memory, and manual memory management should
> > really
> > be done with malloc and free rather than explicitly freeing GC memory. But
> > if
> > you really want to risk blowing your foot off, you can always use destroy
> > to
> > destroy an object in GC memory and core.memory.GC.free to free it.
> > 
> > Also, once we get custom allocators, it should be easier to manually
> > manage
> > memory (e.g. I would assume that it would properly abstract doing malloc
> > and
> > then emplacing the object in that memory so that you do something like
> > allocator!MyObject(args) rather than having to deal with emplace
> > directly).
> 
> Custom allocators will probably be very useful, but if there's one thing
> STL has taught me, it's hard to use them effectively, and in practise,
> nobody ever uses them.

Well, as Andrei said, they're hard, which is why they aren't done yet. Another 
think to think about with regards to C++ though is the fact that its new and 
delete don't having anything to do with a GC, so it has a built-in nice way of 
allocating memory which is managed manually, whereas in D, we're forced to use 
emplace, which is a lot more of a hassle. Even simply having something like 
allocator.make!MyObj(args) and allocator.free(args) would really help out. 
There's no question though that they get hairier when you start having to 
worry about containers and internal allocations and the like. It's a tough 
problem.

> One problem is the implicit allocation functions (array concatenation,
> AA's, etc). How to force those to allocate somewhere else for the scope?

I would fully expect that they use the GC and only the GC as they're language 
constructs, and custom allocators are going to be library constructs. The 
allocators may provide clean ways to do stuff like concatenating arrays using 
their API rather than ~, but if you really want to manipulate arrays with 
slicing and concatenation and whatnot without the GC, I think that you're 
pretty much going to have to create a new type to handle them, which is very 
doable, but it does mean not using the built-in arrays as much, which does 
kind of suck. But for most programs, I would expect that simply managing the 
GC more intelligently for stuff that has to be GC allocated would solve the 
problem nicely. Kiith-Sa and others have managed to quite well at getting the 
GC to work efficiently by managing when it's enabled and gets the chance to run 
and whatnot. You have extremely stringent requirements that may cause problems 
with that (though Kiith-Sa was doing a game of some variety IIRC), but pretty 
much the only way to make it so that built-in stuff that allocates doesn't use 
the GC is to use your own version of druntime.

Kiith-Sa had a good post on how to go about dealing with the GC in performant 
code a while back:

http://forum.dlang.org/post/vbsajlgotanuhmmpnspf@forum.dlang.org

Regardless, we're not going to get away from some language features requiring 
the GC, but they're also not features that exist in C++, so if you really 
can't use them, you still haven't lost anything over C++ (as much as it may 
still suck to not be able to them), and there are still plenty of other great 
features that you can take advantage of.

- Jonathan M Davis