Any takers for http://d.puremagic.com/issues/show_bug.cgi?id=9673?

[Rainer Schuetze]

On 10.03.2013 12:54, Vladimir Panteleev wrote:
> On Sunday, 10 March 2013 at 11:25:13 UTC, Rainer Schuetze wrote:
>>
>>
>> On 10.03.2013 11:32, Vladimir Panteleev wrote:
>>> On Sunday, 10 March 2013 at 10:27:38 UTC, Rainer Schuetze wrote:
>>>> In my experience single file compilation of medium sized projects is
>>>> unacceptably slow. Much slower than what you are used to by similar
>>>> sized C++ projects.
>>>
>>> Even when taking advantage of multiple CPU cores?
>>
>> I don't have support for building on multiple cores, but trying it on
>> visuald itself (48 files) yields
>>
>> - combined compilation    6s
>> - single file compilation 1min4s
>>
>> You'd need a lot of cores to be better off with single file compilation.
>>
>> These are only the plugin files, not anything in the used libraries
>> (about 300 more files). Using dmd compiled with dmc instead of cl
>> makes these times 17s and 1min39s respectively)
>>
>> Almost any change causes a lot of files to be rebuilt (just tried one,
>> took 49s to build).
>
> Do you think it has much to do with that Windows has a larger overhead
> for process creation?

I doubt that causes a significant part of it. I think it's related to 
some files importing the translated Windows-SDK and VS-SDK header files 
(about 8MB of declarations) and these get imported (indirectly) by 
almost any other file.

>
> I've ran some tests on Linux:
>
> ~$ git clone git://github.com/CyberShadow/DFeed.git
> ~$ cd DFeed
> ~/DFeed$ git submodule init
> ~/DFeed$ time rdmd --force --build-only dfeed
> real    0m2.290s
> user    0m1.960s
> sys     0m0.304s
> ~/DFeed$ dmd -o- -v dfeed.d | grep '^import ' | sed 's/.*(\(.*\))/\1/g'
> | grep -v '^/' > all.txt
> ~/DFeed$ time bash -c 'cat all.txt | xargs -n1 dmd -c'
> real    0m16.935s
> user    0m13.837s
> sys     0m2.812s
> ~/DFeed$ time bash -c 'cat all.txt | xargs -n1 -P8 dmd -c'
> real    0m3.703s
> user    0m23.005s
> sys     0m4.412s
>
> (deprecation messages omitted)
>
> I think 2.2s vs. 3.7s is a pretty good result. This was on a 4-core i7 -
> results should be even better with the new 8-cores on the horizon.

Looks pretty ok, but considering the number of modules in dfeed (I count 
about 24) and them being not very large, that makes compilation speed 
for each module about 1 second. It will only be faster if the number of 
modules to compile does not exceed twice the number of cores available. 
I think it does not scale well with increasing numbers of modules.