Regarding hex strings

Marco Leise Marco.Leise at
Thu Oct 18 20:14:44 PDT 2012

Am Thu, 18 Oct 2012 16:31:57 +0200
schrieb "monarch_dodra" <monarchdodra at>:

> On Thursday, 18 October 2012 at 13:15:55 UTC, bearophile wrote:
> > monarch_dodra:
> >
> >> hex! was a very good idea actually, imo.
> >
> > It must scale up to "real world" usages. Try it with a program
> > composed of 3 modules each one containing a 100 KB long string.
> > Then try it with a program with two hundred of medium sized
> > literals, and let's see compilation times and binary sizes.
> >
> > Bye,
> > bearophile
> Hum... The compilation is pretty fast actually, about 1 second, 
> provided it doesn't choke.
> It works for strings up to a length of 400 lines @ 80 chars per 
> line, which result to approximately 16K of data. After that, I 
> get a DMD out of memory error.
> DMD memory usage spikes quite quickly. To compile those 400 lines 
> (16K), I use 800MB of memory (!). If I reach about 1GB, then it 
> crashes.
> I tried using a refAppender instead of ret~, but that changed 
> nothing.
> Kind of weird it would use that much memory though...
> Also, the memory doesn't get released. I can parse a 1x400 Line 
> string, but if I try to parse 3 of them, DMD will choke on the 
> second one. :(

Hehe, I assume most of the regulars know this: DMD used to
use a garbage collector that is disabled. Memory just isn't
freed! Also it has copy on write semantics during CTFE:

int bug6498(int x)
    int n = 0;
    while (n < x)
    return n;
static assert(bug6498(10_000_000)==10_000_000);

--> Fails with an 'out of memory' error.

So, as strange as it sounds, for now try not to write often or
into large blocks. Using this knowledge I was sometimes able
to bring down the memory consumption considerably by caching
recurring concatenations of two strings or to!string calls.

That said, appending single elements to an array may actually
be better than using a fixed-sized one and have DMD duplicate
it on every write. :p

Please remember to give Don a cookie when he manages to change
the compiler to modify in-place where appropriate.


More information about the Digitalmars-d mailing list