Anticipate the usage of Visual Studio 2017 Developer Command Prompt?

[Rainer Schuetze]

On 04.11.2017 12:16, Andre Pany wrote:
> On Saturday, 4 November 2017 at 09:39:02 UTC, Rainer Schuetze wrote:
>>
>>
>> On 03.11.2017 21:51, Andre Pany wrote:
>>> On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
>>>> [...]
>>>
>>> I have an idea which solves several problems.
>>>
>>> Current state:
>>> The dmd windows archive has a folder "bin" with a 32 bit dmd compiler 
>>> which is able to create x86 and x86_64 applications. By default the 
>>> dmd compiler creates 32 bit applications using the optlink link.exe 
>>> contained also in this folder.
>>>
>>> Idea:
>>> Within the folder "bin64" the 64 bit dmd compiler could be shipped 
>>> within the dmd windows archive. I assume the 64 bit dmd compiler is 
>>> able to create x86 and x86_64 applications. In folder "bin64" no 
>>> optlink link.exe should be available.
>>> The usage of either the microsoft link.exe or another linker (maybe 
>>> the one from LDC) is anticipated. The 64 bit dmd compiler could have 
>>> the option -m64 set as default.
>>>
>>> The user can decide whether he wants to use the "stable" dmd compiler 
>>> with usage of optlink or he want to use the 64 bit dmd compiler which 
>>> is open for other linkers than optlink by simply setting his path 
>>> variable to bin or to bin64.
>>>
>>> Using this approach:
>>> - DMD compiler is working out of the box for the VS Developer Command 
>>> Prompt - no need to rename the optlink link.exe
>>> - A 64 bit dmd compiler is shipped which solves issues on large D 
>>> projects
>>>
>>> What do you think?
>>>
>>> Kind regards
>>> André
>>
>> Finding the wrong linker seems to happen quite often, but I wonder why:
>>
>> sc.ini usually has the absolute path to the linker set via the LINKCMD 
>> variable, so finding the correct executable should not be a problem if 
>> the installer has set things up correctly. If extracted from the zip 
>> there are also defaults that might not work but should not find the 
>> wrong linker.
>>
>> Is it the result of trying to fix the sc.ini manually?
>>
>> The problem with overriding the LIB environment variable is that 
>> optlink and the MS linker both use it, but stumble over the libraries 
>> for the other linker due to different file formats.
>>
>> I think the best option forward is to not rely on the installer to 
>> find the correct setup at installation time, but detect the available 
>> linker and its environment before invoking it:
>>
>> - if the environment is already set up via vcvarsall, i.e. 
>> VCINSTALLDIR set, use that to call the approriate linker
>>
>> - if VCINSTALLDIR is not set, use the registry to find the best fit, 
>> i.e. highest available VS version
>>
>> - some way to override the automatism should be available, too.
>>
>> We might also be able to provide some fallback if no VS installation 
>> is found, based on the LLVM linker, but with limited C 
>> runtime/platform library support.
> 
> I would say, by executing vcvarsall the user showed his intention to use 
> the coff libraries and the Microsoft linker for the actual console 
> session instead of the omf libraries.
> 
> I see a disadvantage by detecting the vc installation directory / lib 
> directories. All paths are specific for the vs version. If there is x 
> new vs version we are always behind.
> 
> But Microsoft has a solution for this problem. Vcvarsall sets the linker 
> as first element of the path variable and the lib env variable points to 
> the right folder. This solution will work for old vs version and any 
> upcoming version.

That's what I wrote above, too: the settings from vcvarsall should take 
precedence. The library path for the phobos library must be added, though.

> 
> I set linkcmd to "link.exe". Unfortunately the link.exe in the dmd bin 
> folder has precedence over the path env variable.

This is caused by CreateProcess looking in the directory of the calling 
process first. This could be avoided in dmd by searching the PATH 
folders explicitly before invoking the linker.