Overlapping functionality: IFTI, templates, is-expressions

[BCS]

Russell Lewis wrote:
> BCS wrote:
> 
>>> or!(':',
>>> Parse_block,  /* parses a nonterminal */
>>> chain!("import", array!(IDENT, ','), ';'),
>>> ... many other alternativess ...)
>>> (delegate void(... auto-generated arg types ...)
>>> {
>>> ... process successful parse here ...
>>> });
>>
>>
>> Errk!! Ow, Ow, ow. To much extra syntax! (for my taste) Mine works off 
>> a single string:
> 
> 
> True!  But what I'm attempting to do, at this point in the development, 
> is to develop an as-simple-as-I-can-get-it template parser library.  As 
> I see it, the *next* step is to build a string parser which calls the 
> parser library.  And, if I say so myself, I am, IMHO, getting pretty 
> close to "as dense as you can get without having to parse strings at 
> compile time."

I'll grant that.

I came from the other side; In my system the string parser is an 
integral part of the system. I don't think it could be trivially removed 
(or have bean added after the rest was written).

> 
> Ofc, if you have a parser library which is functional even if it's a 
> little clunky, then hopefully you could use that library to scan input 
> strings, build parse trees, and then generate parsers from that. :)
> 

I don't follow: Are you talking about parsing at run time or compile time?

> 
> So introduce me to your parser a bit.  How do you handle things like:
> * Single character tokens (matching the . operator in D syntax, for
>   instance)
> * Multi-character tokens (matching D keywords)
> * Lexer-recognized tokens (IDENT, NUM, CHAR_LITERAL, etc.)
> 

(attached is an example parser implementation)

Short version; I don't, I don't, and I do.

Longer version. I don't even bother with lexing. That is left completely 
up to the programmer. They can even do it in Lex/Flex if they want. What 
I expect is an object that will give me it's "position" and can be set 
back to that "position" by feeding that value back in. This object is 
then passed back to named functions that the programmer supplys and 
these functions do the lexing or read into a pre lexed stream or whatever.

> Also, how good are you at handling ambiguous grammars?
> 

It doesn't care. It work by matching the first match given so if there 
is ambiguity, it won't know and will still produce something as output.

> How about any built-in way to handle repeated strings of elements, and 
> turn them into arrays?

Yes, mostly. The grammar definitions allow '+', '*' and '?'. But can't 
handle groups except as another non terminal. The annotated terms are 
aggregated into an Object that can return the array of items.

> 
> Finally, how do you define the action-code?
> 

This is done be referencing a function for each disjunct in the rule.

The terminals and actions code for the system (the actual D code the 
user needs to wright) are suppled by defining explicit string 
specializations of the "Terminal" and "Action" templates.

> 
> 
> I'm not so much drawing conclusions ("this sort of grammar can never 
> work") as looking for alternatives ("I wonder if...").  Tell me the 
> details about your design!

My generator will consume the (almost unmodified) D grammar in the spec. 
I actually have a sed script that will extract it for use (it's not done 
but the rest isn't to complex).

I still need to factor out left recursion but I think I can automate 
that. My current project is allowing special cases action rules that 
will make this easy (e.g. process "item items*" as a left sides tree).

Attached is a parser that uses dparse to, at runtime, parse the 
specification grammar that dparse uses at compile time. It might be a 
bit to new for dparse (I have been working on it and don't want to 
commit it in a bad state) but it should give you an idea of how things 
work. BTW the lexer (stuff before about line 210) in there is a bit 
cheesy because I'm lazy and that works. The /real/ guts start around 
line 315.

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: factor.d
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20080320/ea3b6274/attachment.ksh>