Let's stop parser Hell

[Jacob Carlborg]

On 2012-08-01 22:10, Jonathan M Davis wrote:

> It may very well be a good idea to templatize Token on range type. It would be
> nice not to have to templatize it, but that may be the best route to go. The
> main question is whether str is _always_ a slice (or the result of
> takeExactly) of the orignal range. I _think_ that it is, but I'd have to make
> sure of that. If it's not and can't be for whatever reason, then that poses a
> problem. If Token _does_ get templatized, then I believe that R will end up
> being the original type in the case of the various string types or a range
> which has slicing, but it'll be the result of takeExactly(range, len) for
> everything else.

To me a string type would be enough. I don't need support for ranges. 
How about adding a union instead?

enum StringType
{
     utf8,
     utf16,
     utf32
}

struct Token
{
     StringType stringType;

     union
     {
         string strc;
         wstring strw;
         dstring strd;
     }

     @property T str (T = string) ()
     {
         static if (is(T == string))
         {
             assert(stringType == StringType.utf8);
             return strc;
         }
         ...
     }
}

Most use cases would look like this:

string s = token.str;

> I just made str a string to begin with, since it was simple, and I was still
> working on a lot of the initial design and how I was going to go about things.
> If it makes more sense for it to be templated, then it'll be changed so that
> it's templated.

I completely understand that.

-- 
/Jacob Carlborg