D1: Out of memory problems

jicman via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Mon Apr 6 13:22:27 PDT 2015


Greetings.

I am using,

15:32:35.63>dmd
Digital Mars D Compiler v1.046
Copyright (c) 1999-2009 by Digital Mars written by Walter Bright
Documentation: http://www.digitalmars.com/d/1.0/index.html

And I have a program that reads a file into UTF8 and does a 
series of string handling to create reports using an Associative 
Array of Arrays.  Then reads another file and does the same thing 
to each file and creates a report based on word usage, etc.  The 
problem is that the program is not releasing the memory.  Imagine 
this program:

//start
class TUCount
{
   int[char[]] File;
   char[][char[]] Target;
   int Count;
}

void ConsistencyCheck(char[] dir)
{
   TUCount[char[]] aTUs;
   char[][] allfiles = std.file.listdir(dir,"*.txt");
   aTUs = GrabUnits(allfiles);
   PrepareReport(aTUs);
}
TUCount[char[]] GrabUnits(char[][] allfiles)
{
   TUCount[char[]] aTUs;
   foreach (char[] f;allfiles)
   {
     char[] wText = "";
     wText = ReadFileData2UTF8(f, bom); //comes from another 
library and not in this file
                                        //<--Out of memory is 
happening in here...
     while (wText.length > 0)
     {
        // lots of some text handling and update aTUs base on text
     }
   }
}
void main
{
   char[] dir = r"C:\temp\LotsOfTextFiles";
   ConsistencyCheck(dir);
}
//end

The out of memory is happening in the ReadFileData2UTF function.  
All that function does is to read the BOM and read the whole file 
into a variable and returns the UTF8 encoded string.  The problem 
is that apparently, it is reading the files and keeping that data 
there and never releasing it.  The taskmanager just keeps on 
growing and growing, etc.  I know that the aTUs content, which is 
being used to keep track of words, etc., is really low on memory 
usage, and it is not the cause of the huge amount of memory shown 
by the taskmanager.  I have 4G on a Win7 x32.  Any help would be 
appreciated.  Thanks.

josé


More information about the Digitalmars-d-learn mailing list