Latest string_token Code

[Ben Hanson]

== Quote from bearophile (bearophileHUGS at lycos.com)'s article
> Ben Hanson:
> > The underscores thing just comes from the C++ source.
> But once your program works well you can change the variable names a little, or
even before if you have some kind of IDE.
> In D style guide structs and classes need to start with an upper case, in
CamelCase. And variable names are written in camelCase with a starting lower case:
> http://www.digitalmars.com/d/2.0/dstyle.html
> Following a common style guide is important.
I'm happy to change the naming style when things are running.
> > I was recommended that
> > approach, as not wanting to use Reverse Polish Notation (i.e. MFC style),
> I think you mean polish with no reverse :-)
RPN is for compilers isn't it?! I think it was called Hungarian Notation (possibly)...
> > the underscores allow you to have a type the same name as a member var or
local var.
> I don't understand.
> Why can't you write the code like this?
> struct BasicStringToken
> {
>         enum size_t MAX_CHARS = CharT.max + 1;
>         enum size_t START_CHAR = cast(CharT) 0x80 < 0 ? 0x80 : 0;
>         private bool negated = false;
>         private CharT[] charset;
>         this(const bool negated_, ref CharT[] charset_)
>         {
>                 negated = negated_;
>                 charset = charset_;
>         }
> I have kept the underscores in the arguments of the method because they have a
limited scope/life, so they don't add a lot of noise to the whole code.
The code so far isn't a good example of that, but it's generally when you
typedefed a template and then wanted to name a var with the same name as the type.
Regardles, as you pointed out, D does things differently, which is fine.

Regards,

Ben