dmd json file output

[Rainer Schuetze]

On 23.01.2013 06:42, Andrei Alexandrescu wrote:
> On 1/22/13 3:36 PM, Walter Bright wrote:
>> On 1/22/2013 11:46 AM, Andrei Alexandrescu wrote:
>>> On 1/22/13 2:48 AM, Walter Bright wrote:
>>>> On 1/21/2013 10:56 PM, ric wrote:
>>>>> Would it be reasonable to put an option whether to produce the (too)
>>>>> verbose
>>>>> json output or the minimal one?
>>>>
>>>> I'd rather we make a decision.
>>>
>>> Verbose should probably be it.
>>
>> Rationale?
>
> You can always filter out the verboseness with a simple program, but you
> can't add missing information.
>
> If the efficiency of generating json ever comes up, _then_ it's worth
> looking into an option that produces less verbose output directly. For
> now be verbose and let downstream tools filter it out.
>
>
> Andrei
>

I updated dmd from github and had a look at the current json output: 
it's horrible. Below is a random example of a simple function.

- the function parameters are listed three times with different type 
information

- "originalType" seems to be always shown, even though it probably was 
meant to if it is different from "type"

- if the parameter identifiers are listed separately anyway, they should 
not be part of the type while the types do not have to be repeated n the 
actual parameter list

- package and module are specified inconsistently, sometimes as an array 
of strings, sometimes in dot-notation, sometimes not at all.

- types are sometimes shown expanded, sometimes not (e.g. "string")

- template instantiations from imported source files are listed

- functions and template instantiations that are only used at compile 
time are listed

- I appreciate that some missing information has been added, like 
imports and storage class

- renamed imports don't show the original module name

- functions implemented through template mixins are not listed

- surprisingly the average output has only become about 10 times larger 
for a medium sized project like Visual D (73 MB instead of 8 MB). Having 
only std.json available for reading it, I suspect it will definitely 
have an impact on IDE performance, though.

I understand that most of these issues are QOI issues but it also seems 
that there is also a shift in the target usage of the JSON output. It 
was a means for source code browsing with output similar to generated di 
files, while it is now showing everything written into object files 
similar to debug info. Some of this can easily be filtered out (e.g. 
"template instance") but not all (e.g. functions from other modules only 
used in CTFE).

So I think that we should remove excessive bloat (e.g. always specify 
package and module lists in dot notation), make output more consistent 
and avoid listing the same type again and again. If a type is specified 
by its mangled name in declarations, add it to a dictionary at the end 
of the json file in its full verbosity. (I agree core.demangle does not 
help you if you want to do anything more than just getting the pretty 
type string). Please be aware that you will have to document the JSON 
type format in addition to the existing name mangling, though.

Rainer

--------
JSON output for
void setAttribute(Element elem, string attr, string val);

dmd 2.061:
{
"name" : "setAttribute",
"kind" : "function",
"protection" : "public",
"type" : "void(Element elem, string attr, string val)",
"line" : 37}
,


dmd 2.062alpha:
    {
     "name" : "setAttribute",
     "kind" : "function",
     "loc" : {
      "line" : 37
     },
     "module" : {
      "name" : "xmlwrap",
      "kind" : "module",
      "package" : [
       "visuald"
      ],
      "prettyName" : "visuald.xmlwrap"
     },
     "type" : {
      "kind" : "function",
      "pretty" : "void(Element elem, string attr, string val)",
      "returnType" : {
       "kind" : "void",
       "pretty" : "void"
      },
      "parameters" : [
       {
        "name" : "elem",
        "type" : {
         "kind" : "class",
         "pretty" : "std.xml.Element"
        }
       },
       {
        "name" : "attr",
        "type" : {
         "kind" : "darray",
         "pretty" : "string",
         "elementType" : {
          "kind" : "char",
          "pretty" : "immutable(char)",
          "modifiers" : " immutable"
         }
        }
       },
       {
        "name" : "val",
        "type" : {
         "kind" : "darray",
         "pretty" : "string",
         "elementType" : {
          "kind" : "char",
          "pretty" : "immutable(char)",
          "modifiers" : " immutable"
         }
        }
       }
      ]
     },
     "originalType" : {
      "kind" : "function",
      "pretty" : "void(Element elem, string attr, string val)",
      "returnType" : {
       "kind" : "void",
       "pretty" : "void"
      },
      "parameters" : [
       {
        "name" : "elem",
        "type" : {
         "kind" : "identifier",
         "pretty" : "Element",
         "idents" : [],
         "rawIdentifier" : "Element",
         "identifier" : "Element"
        }
       },
       {
        "name" : "attr",
        "type" : {
         "kind" : "identifier",
         "pretty" : "string",
         "idents" : [],
         "rawIdentifier" : "string",
         "identifier" : "string"
        }
       },
       {
        "name" : "val",
        "type" : {
         "kind" : "identifier",
         "pretty" : "string",
         "idents" : [],
         "rawIdentifier" : "string",
         "identifier" : "string"
        }
       }
      ]
     },
     "parameters" : [
      {
       "name" : "elem",
       "kind" : "variable",
       "loc" : {
        "line" : 37
       },
       "module" : {
        "name" : "xmlwrap",
        "kind" : "module",
        "package" : [
         "visuald"
        ],
        "prettyName" : "visuald.xmlwrap"
       },
       "type" : {
        "kind" : "class",
        "pretty" : "std.xml.Element"
       }
      },
      {
       "name" : "attr",
       "kind" : "variable",
       "loc" : {
        "line" : 37
       },
       "module" : {
        "name" : "xmlwrap",
        "kind" : "module",
        "package" : [
         "visuald"
        ],
        "prettyName" : "visuald.xmlwrap"
       },
       "type" : {
        "kind" : "darray",
        "pretty" : "string",
        "elementType" : {
         "kind" : "char",
         "pretty" : "immutable(char)",
         "modifiers" : " immutable"
        }
       }
      },
      {
       "name" : "val",
       "kind" : "variable",
       "loc" : {
        "line" : 37
       },
       "module" : {
        "name" : "xmlwrap",
        "kind" : "module",
        "package" : [
         "visuald"
        ],
        "prettyName" : "visuald.xmlwrap"
       },
       "type" : {
        "kind" : "darray",
        "pretty" : "string",
        "elementType" : {
         "kind" : "char",
         "pretty" : "immutable(char)",
         "modifiers" : " immutable"
        }
       }
      }
     ],
     "endloc" : {
      "line" : 40
     }
    },