Notepad++

[Stewart Gordon]

Sergey Gromov wrote:
<snip>
> Well, you can write a regexp to handle a simple C string.  That is, if
> your regexp is matched against the whole file, which is usually not the
> case.  Otherwise you'll have troubles with C string:
> 
> "foo\
> bar"
> 
> or D string:
> 
> "foo
> bar"

So there is a problem if the highlighter works by matching regexps on a 
line-by-line basis.  But matching regexps over a whole file is no harder 
in principle than matching line-by-line and, when the maximal munch 
principle is never called to action, it can't be much less efficient. 
(The only bit of C or D strings that relies on maximal munch is octal 
escapes.)

> Then you want to highlight string escapes and probably format
> specifiers.  Therefore you need not simple regexps but hierarchies of
> them, and also you need to know where *internals* of the string start
> and end.

Let's just concentrate for the moment on the simple process of finding 
the beginning and end of a string.  Here's a snippet of a TextPad syntax 
file:

StringsSpanLines = Yes
StringStart = "
StringEnd = "
StringEsc = \

A possible snippet of lexer code to handle this (which FAIK might be 
near enough how TP does it):

if (*c == StringStart) {
     beginHighlightString(c);
     for (++c; *c != StringEnd && *c != '\0'
           &&(StringsSpanLines || *c != '\n'); ++c) {
         if (*c == StringEsc) ++c;
     }
     endHighlightString(c+1);
}

It's simple and it should work.  (OK, there are two assumptions made for 
simplicity: that line breaks are normalised to LF, and that the file is 
terminated by at least two null bytes in memory, but you get the idea.)

While it doesn't support highlighting of escapes, I can't see this fact 
as being the reason N++'s developers haven't implemented even this in 
the generic lexer module.  I probably couldn't see it being the reason 
even if the C lexer did highlight escapes (which it doesn't).

> Then you have r"foo" which probably can be handled with regexps.
> 
> Then you have q"/foo/" where "/" can be anything.  Still can be handled
> by extended regexps, even though they won't be regular expressions in
> scientific sense.
> 
> Then you have q"{foo}" where "{" and "}" can be any of ()[]<>{}.
> Regexps cannot translate while substituting, so you must create regexps
> for all possible parens.

Yes, these aspects are more complicated.  Both TP and N++ (out of the 
box, anyway) are probably far from being able to lex D2 properly.  But 
they certainly could do better in supporting D1.  Still, once N++ gains 
access to Scintilla's D lexer, things will certainly be better.

> And of course q"BLAH
> whatever BLAH here
> BLAH", well, probably nice for help texts.
> 
> And these are only strings.  Try to write regexp which treats .__15 as
> number(.__15), .__foo as operator(.), ident(__foo), and 2..3 as
> number(2), operator(..), number(3).
<snip>

We'd need many regexps to handle all possible cases, but a possible set 
to cover these cases and a few others (listed in a possible order of 
priority) is:

\._*[0-9][0-9_]*
([1-9][0-9]*)(\.\.)
[0-9]+\.[0-9]*
[1-9][0-9]*
\.\.
\.
[a-zA-Z_][a-zA-Z0-9_]*

Note the use of capturing groups to handle the 2..3 case.  Each 
capturing group would match a token, while in the other cases the whole 
regexp matches a token.

Stewart.