Are Delimited strings and HereDoc strings just here to suck ?

H. S. Teoh via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Mon Aug 11 13:58:07 PDT 2014


On Mon, Aug 11, 2014 at 10:50:34PM +0200, Philippe Sigaud via Digitalmars-d-learn wrote:
> On Mon, Aug 11, 2014 at 10:09 PM, H. S. Teoh via Digitalmars-d-learn
> <digitalmars-d-learn at puremagic.com> wrote:
> > On Mon, Aug 11, 2014 at 07:47:44PM +0000, Klaus via Digitalmars-d-learn wrote:
> >> I mean when writing a D lexer, you necessarly reach the moment when
> >> you think:
> >>
> >> "Oh no! is this feature just here to suck ?"
> 
> 
> > The crazy variety of ways to write string literals in D, OTOH, *is*
> > a bit over the top, as I found out myself when I also tried writing
> > a D lexer.  :-P
> 
> Out of curiosity, how does a lexer deal with heredocs? It's a sort
> of... user-defined token, right?

In Flex, one way you can implement heredocs is to have a separate "mode"
where the lexer is scanning for the ending string.  So basically you
have a sub-lexer that treats the heredoc as three tokens, one that
defines the ending string for the heredoc (which is never returned to
the caller), one that contains the content of the heredoc, and the
terminating token (also never returned to the caller).

It's not that much different from any other string literal scanning (the
lexer must switch into "string literal mode" where things like \n and \t
have a different meaning than in the program code proper, and it exits
that mode when it encounters the terminating '"'), except that here, the
terminating delimiter is variable.


T

-- 
Real men don't take backups. They put their source on a public FTP-server and let the world mirror it. -- Linus Torvalds


More information about the Digitalmars-d-learn mailing list