[D-runtime] foreach (dchar c; s) is too slow

[Jonathan M Davis]

On Monday, September 30, 2013 20:47:52 Martin Nowak wrote:
> The runtime support for string foreach with decoding has some serious
> performance issues.
> It's currently implemented by creating a delegate for the foreach body
> and calling _aApply*.
> I recently submitted this pull to vibe.d
> https://github.com/rejectedsoftware/vibe.d/pull/327.
> Using the explicit for (auto tmp = s; !tmp.empty; tmp.popFront()) { auto
> c = tmp.front; /*CODE*/ }
> is about 3 times faster.
> I'd like to avoid such hidden performance pitfalls. Any ideas how to
> transition to templated library code?
> For dchar iteration the compiler could simply prefer the range
> interface, but that wouldn't work for wchar (or char with w/dstrings).
> Is it something that should only be added to phobos, e.g. foreach (c;
> s.byChar!dchar())?
> Or should this be a combination of compiler and library support, i.e.
> when an explicit char type is given the compiler
> instantiates a template with that char type.

The range API won't even work on arrays unless std.array has been imported, 
and in the case of strings, it depends on std.utf. So, using the range 
interface underneat the hood for

foreach(dchar c; str)

would require Phobos. I'd argue for simply fixing how foreach for strings works 
so that it doesn't use opApply. Worst case, some druntime-specific functions 
could be written and used to do it, though that would probably require changes 
in the compiler. But if you want to use a range of some kind, just create one 
for each character type, and the fact that Phobos treats strings as ranges of 
dchar should be irrelevant. It does that by having traits like hasLength and 
isNarrowString treat them that way. There's no reason why you can't have a 
range of char or wchar if you want to, and if druntime is wrapping strings in 
ranges for foreach, then it can trivially create a range type which treats 
strings as ranges of whatever character type you want just so long as front 
and popFront do the appropriate conversion. Probably another thing to consider 
in all this is how to deal with foreach_reverse, but that can probably be 
dealt with as well.

I don't particularly like the fact that we're forced to duplicate std.utf stuff 
in druntime in order to handle foreach, but I don't know of any way around 
that short of moving std.utf (or some portion of it) into druntime. It's 
pretty much the same problem that we have with the fact that std.traits isn't 
available in druntime, except that the implementation for decode and stride 
are a bit more complicated than most traits.

- Jonathan M Davis