Best memory management D idioms

[Eugene Wissner via Digitalmars-d-learn]

On Sunday, 5 March 2017 at 20:54:06 UTC, XavierAP wrote:
> I was going to name this thread "SEX!!" but then I thought 
> "best memory management" would get me more reads ;) Anyway now 
> that I have your attention...
> What I want to learn (not debate) is the currently available 
> types, idioms etc. whenever one wants deterministic memory 
> management. Please do not derail it debating how often 
> deterministic should be preferred to GC or not. Just, whenever 
> one should happen to require it, what are the available means? 
> And how do they compare in your daily use, if any? If you want 
> to post your code samples using special types for manual memory 
> management, that would be great.
>
> AFAIK (from here on please correct me wherever I'm wrong) the 
> original D design was, if you don't want to use GC, then 
> malloc() and free() are available from std.c. Pretty solid. I 
> guess the downside is less nice syntax than new/delete, and 
> having to check the returned value instead of exceptions. I 
> guess these were the original reasons why C++ introduced 
> new/delete but I've never been sure.
>
> Then from this nice summary [1] I've learned about the 
> existence of new libraries and Phobos modules: std.typecons, 
> Dlib, and std.experimental.allocator. Sadly in this department 
> D starts to look a bit like C++ in that there are too many 
> possible ways to do one certain thing, and what's worse none of 
> them is the "standard" way, and none of them is deprecated atm 
> either. I've just taken a quick look at them, and I was 
> wondering how many people prefer either, and what are their 
> reasons and their experience.
>
> dlib.core.memory and dlib.memory lack documentation, but 
> according to this wiki page [2] I found, dlib defines 
> New/Delete substitutes without GC a-la-C++, with the nice 
> addition of a "memoryDebug" version (how ironclad is this to 
> debug every memory leak?)
>
> From std.typecons what caught my eye first is scoped() and 
> Unique. std.experimental.allocator sounded quite, well, 
> experimental or advanced, so I stopped reading before trying to 
> wrap my head around all of it. Should I take another look?
>
> scoped() seems to work nicely for auto variables, and if I 
> understood it right, not only it provides deterministic 
> management, but allocates statically/in the stack, so it is 
> like C++ without pointers right? Looking into the 
> implementation, I just hope most of that horribly unsafe 
> casting can be taken care of at compile time. The whole thing 
> looks a bit obscure under the hood and in its usage: auto is 
> mandatory or else allocation doesn't hold, but even reflection 
> cannot tell the different at runtime between T and 
> typeof(scoped!T) //eew. Unfortunately this also makes scoped() 
> extremely unwieldy for member variables: their type has to be 
> explicitly declared as typeof(scoped!T), and they cannot be 
> initialized at the declaration. To me this looks like scoped() 
> could be useful in some cases but it looks hardly recommendable 
> to the same extent as the analogous C++ idiom.
>
> Then Unique seems to be analogous to C++ unique_ptr, fair 
> enough... Or are there significant differences? Your experience?
>
> And am I right in assuming that scoped() and Unique (and 
> dlib.core.memory) prevent the GC from monitoring the memory 
> they manage (just like malloc?), thus also saving those few 
> cycles? This I haven't seen clearly stated in the documentation.
>
>
> [1] 
> http://forum.dlang.org/post/stohzfatiwjzemqojeol@forum.dlang.org
> [2] 
> https://github.com/gecko0307/dlib/wiki/Manual-Memory-Management

The memory management in D is becoming a mess. Yes, D was 
developed with the GC in mind and the attempts to make it usable 
without GC came later. Now std has functions that allocate with 
GC, there're containers that use malloc/free directly or 
reference counting for the internal storage, and there is 
std.experimental.allocator. And it doesn't really get better. 
There is also some effort to add reference counting directly into 
the language. I really fear we will have soon signatures like 
"void myfunc() @safe @nogc @norc..".
Stuff like RefCounted or Unique are similar to C++ analogues, but 
not the same. They throw exceptions allocated with GC, factory 
methods (like Unique.create) use GC to create the object.
Also dlib's memory management is a nightmare: some stuff uses 
"new" and GC, some "New" and "Delete". Some functions allocate 
memory and returns it and you never know if it will be collected 
or you should free it, you have to look into the source code each 
time to see what the function does internally, otherwise you will 
end up with memory leaks or segmentation faults. dlib has a lot 
of outdated code that isn't easy to update.