(gcc returns mem) Re: When is it time for a 1.0 feature freeze?

[Bruno Medeiros]

Walter Bright wrote:
> Bruno Medeiros wrote:
>> I think Serg is right. I don't know what program he used to monitor  
>> the mem usage ("process explorer", is that the Windows Task Manager?), 
>> but my test confirms what he's saying.
>>
>> I made a small C program that allocs 100Mb and then frees it. I 
>> monitored it with Windows XP Task Manager process list, looking at the 
>> fields "Mem Usage" (physical mem) and "VM Size". Compiling with GCC 
>> (3.2.3), it does free back the memory to the OS (VM Size decreases 
>> after the free). I tried the program with DMC but the same does not 
>> happen: the memory is not returned after the free. I don't have visual 
>> C available right now so I didn't try that one.
>>
>> After a google search:
>> http://www.gnu.org/software/libc/manual/html_node/Freeing-after-Malloc.html 
>>
>> "Occasionally, free can actually return memory to the operating system 
>> and make the process smaller."
>> Evidently the Windows version of libc also does the same. And seems 
>> it's smart enough to return the mem not just when the top of the heap 
>> is all free (what I was expecting) but also with free pages in the 
>> middle of the heap. (the test program allocs two segments and frees 
>> the first only)
> 
> What your program is doing is allocating a gigantic chunk. It is a 
> reasonable thing for new() to have a fork in it, and for gigantic chunks 
> allocate/free them by calling the OS directly and not attempting to 
> manage it. Thus you would see the behavior you see.
> 
> Try allocating a large number of small chunks, and free them.

Hum, I modified the program and tried 4 more tests:

Allocating 10000 (sequential) chunks of size 10000 bytes. Free them all.
-> All the 100Mb of memory is returned to the OS.

Allocating 100000 (sequential) chunks of size 1000 bytes. Free them all.
-> All the 100Mb of memory is returned to the OS.

Allocating 10000 (sequential) chunks of size 10000 bytes. Free only half 
of them, the even numbered ones, so that the total freed memory is not 
contiguous.
-> Of the 50Mb memory free'd, about 30Mb of memory is returned to the 
OS. Expected due to page segmentantion/rounding.

Allocating 100000 (sequential) chunks of size 1000 bytes. Again free 
only the even numbered ones.
-> 50Mb memory is free'd, but no memory is returned to the OS. Expected 
due to page segmentantion/rounding.

So it seems glibc does its best, it returns any page if it is all free. 
(hum, and I'm curious to what the results in VC++ are, if anyone tries 
it, do post the results)

------ test program ------
#include <stdio.h>
#include <assert.h>

#define CHUNKSIZE 10000
#define NUMCHUNKS 10000
#define CHUNKINC 1 //use CHUNKINC 2 for even numbered freeing

int main() {
	char buf[666];
	int i;
	char* ptrs[NUMCHUNKS];
	
	printf("NumChunks: %d Size: %d Inc: %d\n", NUMCHUNKS, CHUNKSIZE, 
CHUNKINC);
	fflush(stdout); gets(buf);
	
	for(i = 0; i < NUMCHUNKS; ++i) {
		ptrs[i] = (char*) malloc(CHUNKSIZE);
		assert(ptrs[i] != NULL);
	}

	char* ptrs2[NUMCHUNKS];

	for(i = 0; i < NUMCHUNKS; ++i) {
		ptrs2[i] = (char*) malloc(CHUNKSIZE);
		assert(ptrs2[i] != NULL);
		assert(ptrs2[i] > ptrs[NUMCHUNKS-1]);
	}
	printf("Mem Allocated\n"); fflush(stdout); gets(buf);
	
/*	for(i = 0; i < NUMCHUNKS; i++)
		ptrs[i][0] = 'X';
	printf("Memory Writen\n"); fflush(stdout);	gets(buf);
*/

	for(i = 0; i < NUMCHUNKS; i += CHUNKINC) {
		free(ptrs[i]);
	}
	printf("Mem free'd\n"); fflush(stdout); gets(buf);
}

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D