[Issue 8203] Use of std.regex.match() generates "not enough preallocated memory" error

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=8203



--- Comment #6 from Dmitry Olshansky <dmitry.olsh at gmail.com> 2012-06-07 04:35:32 PDT ---
I've studied it a bit, and here is the details:
it only happens, when re-running the same match object many times:

foreach(v; match(...)) // no bug
vs
auto m = match(....)
foreach(v; m) //does run out of memory

In your case I see from comments that you try hard to do eager evalutaion, and
first find all matches then work through two arrays of them. Yet it's not what
program does, it still performes N*M regex searches because 
auto uniCapturesNew = match(uniFileOld, regex(...));

just starts the engine and finds 1st match. Then you copy engine state on each
iteration of nested loop (this copy operation is bogus apparently) and run
engine till all matches are found. Next iteration of loop  - another copy.

So in your case I strongly suggest to do this magic recipe, that work for all
lazy ranges:
auto allMatches = array(match(....);

and work with arrays from now on.


Anyway, the root cause is now clear and I've reduced it to:

import std.regex;
string data = "
NAME   = XPAW01_STA:STATION
NAME   = XPAW01_STA
";
// Main function
void main(){
    auto uniFileOld = data;    
    auto uniCapturesNew = match(uniFileOld, regex(r"^NAME   =
(?P<comp>[a-zA-Z0-9_]+):*(?P<blk>[a-zA-Z0-9_]*)","gm"));

    for(int i=0; i<20; i++)
  { foreach (matchNew; uniCapturesNew) {} }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------