[Issue 16039] New: DMD produces huge binary when using a complex chain of lazy range algorithms

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16039

          Issue ID: 16039
           Summary: DMD produces huge binary when using a complex chain of
                    lazy range algorithms
           Product: D
           Version: D2
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P1
         Component: dmd
          Assignee: nobody at puremagic.com
          Reporter: georgid at outlook.com

Created attachment 1597
  --> https://issues.dlang.org/attachment.cgi?id=1597&action=edit
sample code that reproduces the issue

While working on a D project which heavily uses the lazy algorithms for ranges
I noticed a sudden huge increase in the compilation time and the produced
binary size.

I chased down the offending change to just a small change in one line of the
code which made the resulting binary to jump from 18MB to over 400MB.

The code that produces the large binary actually failed to link using DMD on
osx (works on linux). The link command just consumes all the CPU and never
completes. Using LDC on osx just failed to compile. ldc was using all the CPU
and never completing.

There is also a very big difference between passing -unittest and not passing
it.

I was able to shrink the code down to the following:

import std.range.primitives;
import std.range : chain, choose, repeat, chooseAmong, takeOne;
import std.algorithm.iteration : joiner , map;
import std.range.interfaces : InputRange, inputRangeObject;
import std.conv : text;

struct AType {
   string id;
   Annotation[] annotations;
   TypeArgument[] typeArguments;

}

struct Annotation {
   string type;
}

struct TypeArgument {
   AType refType;
}

struct PrimaryValue {
   int type;
}

struct FieldDecl {
   AType type;
   string id;
   PrimaryValue[] values;
}

struct MethodDecl {
   string name;
   AType returnType;
   FieldDecl[] params;
}

auto toCharsArray(R)(R r, string sep = "\n", string tail = "\n")
         if (isInputRange!R) {
    return choose(r.empty, "", chain(r.joiner(sep), tail));
}


auto toChars(Annotation uda) {
   return chain("@", uda.type);
}

InputRange!(ElementType!string) toChars(TypeArgument ta) {
   auto r = chain("", ta.refType.id);
   with (ta.refType) {
      auto tArgs = choose(typeArguments.empty, "",
                          chain("!(", typeArguments.map!(toChars).joiner(", "),
")"));

      return chain(id, tArgs).inputRangeObject();
   }
}

auto toChars(AType t) {
   auto udas = t.annotations.map!(toChars).toCharsArray();
   auto tArgs = choose(t.typeArguments.empty, "",
                       chain("!(", t.typeArguments.map!(toChars).joiner(", "),
")"));

   return chain(udas, t.id, tArgs);
}

/// PrimaryValue
auto toChars(PrimaryValue pv) {
   return chooseAmong(pv.type, "val", "val");
}

auto toCharsSingleOrArray(R)(R r) if (isInputRange!R) {
   return choose(r.length == 1,
                 r.takeOne.map!(toChars).joiner(),
                 chain("[", r.map!(toChars).joiner(", "), "]")
                 );
}

auto toCharsBare(FieldDecl f) {
   auto def = chain(f.type.toChars, " ", f.id);

   return choose(f.values.empty,
                 def,
                 chain(def, " = ", toCharsSingleOrArray(f.values)));
}

auto toChars(FieldDecl f) {
   return chain("", f.toCharsBare());
}

auto toChars(MethodDecl m) {
   //Just changing the line bellow to have map!(toChars) reduces the binary
size
   //to 18MB from 403MB
   auto prms = chain("(", m.params.map!(toCharsBare).toCharsArray(", ", ""),
")");
   return chain(" ", m.name, prms, ";");
}

unittest {
   AType t1 = {id : "Foo"};
   FieldDecl f1 = {type : t1, id : "f1"};

   MethodDecl m1 = {name : "m", returnType : t1 };
   assert(!m1.toChars().text().empty); // removing this line drops the size by
half
}

void main() { }

I am compiling this with:

# dmd -g -debug -unittest


I have marked the line that needs to be changed for the code size to drop
significantly. What is interesting is that the using toChars should be more
complex than using toCharsBare since the former calls the latter but the binary
size just explodes when using toCharsBare.

--