InvalidMemoryOperationError from File.byLine()?

Tue Jun 2 12:57:27 PDT 2015

On Tuesday, 2 June 2015 at 19:35:04 UTC, Alex Parrill wrote:
> On Tuesday, 2 June 2015 at 19:18:15 UTC, Stiff wrote:
>> Hi,
>>
>> I'm reading in a rather large text file (~25Mb) line by line, 
>> where I don't need to hang on to a line for more than one 
>> iteration through my loop. I'm consistently getting an 
>> InvalidMemoryOperationError on my 2,547th iteration, and based 
>> on console outputs, I'm fairly certain it's coming from the 
>> byLine() method. To be clear, I'm just doing a simple:
>>
>> foreach (line; file.byLine()){ ... }
>>
>> Am I possibly screwing something up? I haven't declared any 
>> classes or written any destructors. Is this a bug? If so, is 
>> there a workaround?
>>
>> Thanks!
>
> Can you provide a minimal test case? I.e. as short of a program 
> and input file as you can get that still has the error?

Here's the least code I can reproduce it with:

//appropriate imports here

void main(string[] args)
{
	auto groupFile = new File(args[1]);
	int groupNum = 0;
	foreach (groupLine; groupFile.byLine()){
		writeln(groupNum);
		groupNum++;
	}
}

Data (i.e. groupFile, 2 lines):

  41  7,        144 0,  473 2,  730 1,  229 6,  333 11,  961 15,  
856 16,  20 17,  165 18,  200 19,  395 20,  939 21,  240 24,  760 
24,  434 29,  169 30,  718 30,  845 32,  942 33,  414 35,  889 
36,  944 36,  918 37,  891 38,  976 38,  325 40,  840 40,  884 
40,  931 40,  991 40,  690 41,  802 41,  138 43,  827 43,  934 
43,  886 44,  894 47,  979 53,  892 60,  225 63,  858 67,         
  144 229 3 2,   473 229 1 2,   730 229 2 2,   144 333 6 2,   730 
333 5 2,   144 961 8 2,   473 961 5 2,   144 856 11 2,   473 856 
9 2,   473 20 10 2,   730 20 11 2,   144 165 14 2,   473 165 9 2, 
   229 165 3 3,   730 200 12 2,   229 200 6 3,   473 395 13 2,   
730 395 14 2,   333 939 3 3,   144 939 13 3,   144 240 18 2,   
473 240 15 2,   144 760 19 2,   165 760 1 4,   165 434 5 4,   229 
434 16 4,   395 169 9 3,   333 169 12 3,   229 169 16 3,   20 718 
7 3,   395 718 3 3,   200 845 8 4,   395 845 7 4,   395 942 6 3,  
  333 942 14 3,   165 414 15 4,   20 414 13 4,   760 414 6 5,   
240 889 5 3,   20 889 11 3,   165 944 13 4,   718 944 1 4,   240 
918 8 3,   718 918 2 4,   229 918 17 4,   165 891 17 4,   760 891 
9 5,   169 891 2 5,   165 976 15 4,   434 976 4 5,   200 325 15 
4,   20 325 16 4,   20 840 18 3,   169 840 5 4,   200 840 8 4,   
414 884 3 6,   395 884 14 6,   165 884 15 6,   169 931 5 4,   718 
931 5 4,   200 991 10 4,   165 991 10 4,   414 690 0 6,   200 690 
15 6,   165 802 19 4,   718 802 5 4,   240 802 9 4,   200 138 13 
4,   760 138 8 5,   414 827 5 6,   169 827 7 6,   760 827 12 6,   
200 934 19 4,   414 934 3 6,   434 934 0 6,   240 886 15 3,   169 
886 9 4,   718 886 5 4,   434 894 13 5,   414 894 7 6,   690 979 
9 7,   138 979 5 7,   169 979 17 7,   138 892 9 6,   325 892 11 
6,   690 225 15 7,   138 225 12 7,   690 858 19 7,   325 858 19 7,
  47  8,        144 4,  557 0,  730 5,  229 10,  333 15,  86 17,  
200 23,  184 24,  939 25,  355 26,  169 34,  287 34,  1 37,  138 
41,  949 41,  119 42,  51 46,  356 47,  884 50,  873 51,  783 52, 
  346 53,  900 53,  594 54,  110 55,  925 56,  236 57,  856 60,  
374 61,  840 64,  993 64,  828 65,  858 66,  548 67,  992 68,  
525 70,  563 70,  709 72,  614 74,  746 76,  278 82,  961 83,  
959 85,  509 87,  962 91,  448 96,  256 97,          144 229 3 2, 
   557 229 7 2,   730 229 2 2,   144 333 6 2,   730 333 5 2,   229 
86 0 3,   557 86 9 3,   730 200 12 2,   229 200 6 3,   333 184 2 
3,   144 184 12 3,   333 939 3 3,   144 939 13 3,   730 355 15 2, 
   86 355 2 4,   333 169 12 3,   229 169 16 3,   200 287 9 4,   86 
287 11 4,   229 287 17 4,   86 1 13 4,   200 1 6 4,   200 138 13 
4,   355 138 10 5,   200 949 11 4,   86 949 16 4,   287 119 2 5,  
  169 119 1 5,   200 51 18 4,   184 51 17 4,   184 356 17 4,   169 
356 6 4,   169 884 14 4,   355 884 17 5,   287 884 8 5,   119 873 
6 6,   356 873 1 6,   51 873 1 6,   138 783 0 6,   200 783 17 6,  
  169 346 14 4,   287 346 14 5,   287 900 12 5,   138 900 4 6,   
287 594 14 5,   169 594 13 5,   86 594 17 5,   287 110 14 5,   
355 110 16 5,   184 110 16 5,   169 925 17 4,   51 925 5 5,   287 
925 3 5,   169 236 16 4,   51 236 3 5,   346 856 3 6,   110 856 1 
6,   783 856 1 7,   1 856 5 7,   356 374 7 5,   287 374 19 5,   1 
840 19 5,   356 840 8 5,   51 993 11 5,   346 993 3 6,   51 828 
16 5,   783 828 10 7,   594 828 7 7,   594 858 5 6,   51 858 12 
6,   1 858 16 6,   356 548 14 5,   110 548 5 6,   783 992 11 7,   
346 992 10 7,   51 992 11 7,   346 525 12 6,   110 525 10 6,   
374 563 3 6,   346 563 10 6,   783 709 13 7,   374 709 3 7,   346 
614 15 6,   110 614 12 6,   594 746 16 6,   110 746 14 6,   525 
278 6 7,   236 278 18 7,   709 961 5 8,   746 961 1 8,   374 961 
9 8,   783 961 12 8,   525 959 12 7,   614 959 3 7,   746 959 0 
7,   346 959 14 7,   746 509 3 7,   374 509 17 7,   548 962 19 7, 
   563 962 16 7,   614 962 7 7,   614 448 16 7,   525 448 19 7,   
709 256 19 8,   614 256 16 8,

Sorry for the mess of data, I'm not sure how to attach something.

It looks like a data-dependent problem. So is there another 
method I can use, do I need to punt on that line, or can I alter 
the data in some way to make it not crash?