Significant GC performance penalty
jerro
a at a.com
Sun Dec 16 01:28:02 PST 2012
On Sunday, 16 December 2012 at 07:47:48 UTC, Rob T wrote:
> On Sunday, 16 December 2012 at 05:37:57 UTC, SomeDude wrote:
>>
>> Isn't the memory management completely negligible when
>> compared to the database access here ?
>
> Here are the details ...
>
> My test run selects and returns 206,085 records with 14 fields
> per record.
>
> With all dynamic memory allocations disabled that are used to
> create the data structure containing the returned rows, a run
> takes 5 seconds. This does not return any data, but it runs
> exactly through all the records in the same way but returns to
> a temporary stack allocated value of appropriate type.
>
> If I disable the GC before the run and re-enable it immediately
> after, it takes 7 seconds. I presume a full 2 seconds are used
> to disable and re-enable the GC which seems like a lot of time.
>
> With all dynamic memory allocations enabled that are used to
> create the data structure containing the returned rows, a run
> takes 28 seconds. In this case, all 206K records are returned
> in a dynamically generate list.
>
> If I disable the GC before the run and re-enable it immediately
> after, it takes 11 seconds. Since a full 2 seconds are used to
> disable and re-enable the GC, then 9 seconds are used, and
> since 5 seconds are used without memory allocations, the
> allocations are using 4 seconds, but I'm doing a lot of
> allocations.
>
> In my case, the structure is dynamically generated by
> allocating each individual field for each record returned, so
> there's 206,085 records x 14 fields = 2,885,190 allocations
> being performed. I can cut the individual allocations down to
> 206,000 by allocating the full record in one shot, however this
> is a stress test designed to work D as hard as possible and
> compare it with an identically stressed C++ version.
>
> Both the D and C++ versions perform identically with the GC
> disabled and subtracting the 2 seconds from the D version to
> remove the time used up by enabling and disabling the GC during
> and after the run.
>
> I wonder why 2 seconds are used to disable and enable the GC?
> That seems like a very large amount of time. If I select only
> 5,000 records, the time to disable and enable the GC drops
> significantly to negligible levels and it takes the same amount
> of time per run with GC disabled & enabled, or with GC left
> enabled all the time.
>
> During all tests, I do not run out of free RAM, and at no point
> does the memory go to swap.
>
> --rt
Adding and subtracting times like this doesn't give very reliable
results. If you want to know how much time is taken by different
parts of code, I suggest you use a profiler.
More information about the Digitalmars-d
mailing list