Anticipate the usage of Visual Studio 2017 Developer Command Prompt?

[Rainer Schuetze]

On 03.11.2017 21:51, Andre Pany wrote:
> On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
>> Hi,
>>
>> Visual Studio has a batch file which sets all needed environment 
>> variables for the Microsoft linker (LIB environment variables points 
>> to all necessary folders). Even the folder containing the link.exe for 
>> the specified architecture (x86, amd64) is added to the path variable 
>> as first path.
>>
>>> vsdevcmd -arch=amd64
>>
>> As the sc.ini targets the windows development (visuals studio) after 
>> calling the developer command prompt, dmd should be able to compile 
>> -m32mscoff, -m64 out of the box without the need to change anything 
>> within sc.ini.
>>
>> There is even a comment in sc.ini which points in this direction:
>> ; Add the library subdirectories of all VC and Windows SDK versions so 
>> things
>> ; just work for users using dmd from the VS Command Prompt
>>
>> Unfortunately it does not work for various reasons:
>> - optlink has the filename link.exe which conflicts with the filename 
>> of the microsoft linker
>> - from my investigation I have the assumption that even if an 
>> environment variable LIB exists, the value is discarded by the sc.ini 
>> logic. I tried hard, but it seems existing environment variable LIB 
>> cannot be used in sc.ini
>>
>> If you can use the windows installer, it will set the right values in 
>> sc.ini. But there are scenarios in which you cannot use the windows 
>> installer but have to use the dmd zip archive (build infrastructure 
>> scripts).
>>
>> In my scenario it seems the only possibility is to adapt my build 
>> infrastructure scripts to:
>> - extract dmd archive
>> - delete link.exe (optlink)
>> - modify the sc.ini
>>
>> That is a solution for my case, but my gut feeling is, it should be 
>> solved in general
>> to ease the usage of the microsoft linker.
>>
>> I thought it was planned to rename the optlink link.exe to optlink.exe?
>>
>> Kind regards
>> André
> 
> I have an idea which solves several problems.
> 
> Current state:
> The dmd windows archive has a folder "bin" with a 32 bit dmd compiler 
> which is able to create x86 and x86_64 applications. By default the dmd 
> compiler creates 32 bit applications using the optlink link.exe 
> contained also in this folder.
> 
> Idea:
> Within the folder "bin64" the 64 bit dmd compiler could be shipped 
> within the dmd windows archive. I assume the 64 bit dmd compiler is able 
> to create x86 and x86_64 applications. In folder "bin64" no optlink 
> link.exe should be available.
> The usage of either the microsoft link.exe or another linker (maybe the 
> one from LDC) is anticipated. The 64 bit dmd compiler could have the 
> option -m64 set as default.
> 
> The user can decide whether he wants to use the "stable" dmd compiler 
> with usage of optlink or he want to use the 64 bit dmd compiler which is 
> open for other linkers than optlink by simply setting his path variable 
> to bin or to bin64.
> 
> Using this approach:
> - DMD compiler is working out of the box for the VS Developer Command 
> Prompt - no need to rename the optlink link.exe
> - A 64 bit dmd compiler is shipped which solves issues on large D projects
> 
> What do you think?
> 
> Kind regards
> André

Finding the wrong linker seems to happen quite often, but I wonder why:

sc.ini usually has the absolute path to the linker set via the LINKCMD 
variable, so finding the correct executable should not be a problem if 
the installer has set things up correctly. If extracted from the zip 
there are also defaults that might not work but should not find the 
wrong linker.

Is it the result of trying to fix the sc.ini manually?

The problem with overriding the LIB environment variable is that optlink 
and the MS linker both use it, but stumble over the libraries for the 
other linker due to different file formats.

I think the best option forward is to not rely on the installer to find 
the correct setup at installation time, but detect the available linker 
and its environment before invoking it:

- if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR 
set, use that to call the approriate linker

- if VCINSTALLDIR is not set, use the registry to find the best fit, 
i.e. highest available VS version

- some way to override the automatism should be available, too.

We might also be able to provide some fallback if no VS installation is 
found, based on the LLVM linker, but with limited C runtime/platform 
library support.