compile-time regex redux

[Andrei Alexandrescu (See Website For Email)]

Sean Kelly wrote:
> Walter Bright wrote:
>  >
>> At some point, this will prove a barrier to large scale use of this 
>> feature.
> 
> I agree, though I'm not sure this feature will see large scale use 
> either way.  Template metaprogramming is still very uncommon outside of 
> library code.

If we want to make D a language for the future, we must thoroughly rid 
ourselves of such a view.

>> Andrei suggested using compile time regular expressions to shoulder 
>> much of the burden, reducing parsing of any particular token to one 
>> instantiation.
>>
>> The last time I introduced core regular expressions into D, it was 
>> soundly rejected by the community and was withdrawn, and for good 
>> reasons.
>>
>> But I think we now have good reasons to revisit this, at least for 
>> compile time use only. For example:
>>
>>     ("aa|b" ~~ "ababb") would evaluate to "ab"
>>
>> I expect one would generally only see this kind of thing inside 
>> templates, not user code.
> 
> I agree that this would eliminate the need for a lot of template library 
> code and would speed compilation for applications using such techniques. 
>  I am still unsure whether this is sufficient to warrant its inclusion 
> to the language, but I'm not strongly opposed to the idea.  However, for 
> this to be useful I'd like to reiterate that I would want some way to 
> continue parsing after the match point.  The most obvious would be to 
> return an index/string pair where the index contains the position of the 
> match in the source string, or as you mentioned, perhaps an array 
> consisting of three slices: the source string preceding the match, the 
> match itself, and the source string following the match.

Parens will allow grouping much like in Perl. If a regex contains 
groupings, then the result will be a compile-time array with the 
matches. All you have to do then is to group subparts appropriately, e.g.:

("templated regex rocks" ~~ "([a-z]+) +(.*)")

returns a compile-time array ["templated", "regex rocks"].


Andrei