Looking for champion - std.lang.d.lex

[Nick Sabalausky]

"dennis luehring" <dl.soluz at gmx.net> wrote in message 
news:ia6s3b$1q90$1 at digitalmars.com...
> Am 26.10.2010 16:48, schrieb dennis luehring:
>> Am 26.10.2010 15:55, schrieb Leandro Lucarella:
>>
>> yupp - Spirit feels right on the integration-side, but becomes more and
>> more evil when stuff gets bigger

Goldie (and any GOLD-based system, really) should scale up pretty well. The 
only possible scaling-up issues would be:

1. Splitting a large grammar across multiple files is not yet supported (and 
if I do add support for that in Goldie, and I may, then the "GOLD Parser 
Builder" IDE wouldn't know how to handle it).

2. Somewhat related to #1, grammars composed of multiple languages (such as 
classic-style ASP, or anything that involves a preprocessing step that 
hasn't already been done) aren't really supported yet. Spirit 1.x should be 
able to handle that, at least in some cases. I think Spirit 2.x's separation 
of lexing and parsing may have some trouble with it though.

3. I haven't had a chance to add any sort of character set optimization yet, 
so grammars that allow a large amount of Unicode characters will probably be 
slow to generate into tables and slow to lex. At least until I get around to 
taking care of that.

I've never actually used Spirit, but its scaling up issues do seem to be a 
fairly fundamental issue with it's design (particularly so since it's C++). 
Although they do say on their site that some C++ compilers can handle Spirit 
without compile time growing exponentially in relation to grammar 
complexity.

>>
>> a compiletime-ebnf-script parser would do better, especially when
>> the ebnf-script comes through compiletime-file-include and can be
>> used/developed from outside in an ide like gold parsers
>>

There's one problem with doing things via CTFE that us D folks often 
overlook: You can't use a build tool like make/rake/scons to detect when 
that particular data doesn't need to be recomputed and can thus be skipped. 
(Although it may be possible to manually make it work that way *if* CTFE 
gains the ability to access the filesystem.)

I'm not opposed to the idea of making Goldie's compiling-a-grammar (ie, 
"process a grammar into the appropriate tables") ctfe-able, but it does 
already work in a way that you only need to compile a grammar into tables 
when you change the grammar (and changing a grammar is needed less 
frequently in Goldie than in Spirit because in Goldie no processing code is 
ever embedded into the grammar.).

>
> that combined with compiletime-features something like the bsn-parse do
>
> http://code.google.com/p/bsn-goldparser/
>
> i think this all is very very doable in D

Yea, I was pretty impressed with BSN. I definitely want to do something like 
that for Goldie, but I have a somewhat different idea in mind: I'm thinking 
of enhancing the grammar definition language so that all the information on 
how to construct an AST is right there in the grammar definition itself, and 
can thus be completely automated by Goldie. This would be in line with 
GOLD's philosophy and benefits of keeping the grammar definition separate 
from the processing code. And it would also be a step towards the idea I've 
had in mind since before Goldie was Goldie of being able to automate (or 
partially automate) generalized language translation/transformation.