Writing a Parser

[Alan Knowles]

Yes, normally when doing OO patterns for this

class Parser extends Tokenizer {...}

either that or
class Parser {
	Tokenizer reader;
	parser code....
}

Recommended reading
- the parser in DMD's source (not DMDScript) is quite nice - it's a hand 
coded one and is quite easy to understand.

This is interesting from the perspective of using function pointers to 
do pattern testing (it's horribly borked from the perspective that it's 
really the tokenizer..)
http://www.dsource.org/projects/leds/browser/trunk/src/language/php/Parser.d

If you dont want a hand coded parser, have a look at csLex - It's what 
mono used, and is pretty trivial to modify, and retarget to generate 
other language code (eg. D code...)

 From someone who's written far too many parsers ;)
Regards
Alan




Jarrett Billingsley wrote:
> "Dan" <murpsoft at hotmail.com> wrote in message 
> news:flmtrv$2jrn$1 at digitalmars.com...
>> I've been messing with how to write a parser, and so far I've played with 
>> numerous patterns before eventually wanting to cry.
>>
>> At the moment, I'm trying recursive descent parsing.
>>
>> The problem is that I've realized I'm duplicating huge volumes of code to 
>> cope with the tristate decision of { unexpected, allow, require } for any 
>> given token.
>>
>> For example, to consume a for loop, you consume something similar to
>> /for\s*\((.*?)\)\s*\{(.*?)\}/
>>
>> I have it doing that, but my soul feels heavy with the masses of looped 
>> switches it's doing.  Is there any way to ease the pain?
> 
> Separate tokenization and syntax parsing?  It makes things a hell of a lot 
> easier.  You don't even necessarily have to tokenize the source entirely 
> before parsing; just have a lexer which lexes tokens out of the source on 
> demand.  The syntax parsing is then unencumbered from dealing with the raw 
> source and just has to do stuff like "expect 'for', expect left-paren, 
> expect (your condition), expect right-paren" etc. 
> 
>