On processors for D ~ decoupling

Fri Apr 7 00:56:20 PDT 2006

kris wrote:
> Yes, they are useful. And I'm not trying to be a jerk by pointing out 
> that the D runtime is missing some much needed TLC (to put it very 
> nicely). Why do you think Ares exists anyway?

I'm trying to understand.

>> All those other names are are the static data. Things like:
>>
>> const dchar LS = '\u2028';      /// UTF line separator
>> const dchar PS = '\u2029';      /// UTF paragraph separator
>>
>> I submit that they aren't significant. 
> Yes, you're right. Unless they're huge tables (such as the Unicode 
> character map; oh wait; is that linked by default? :)

If you're talking about the one in std.uni, it took me a minute to 
figure that one out. The reason it's pulled in is  because of an error 
in the compiler, where lambda functions should be generated as comdats, 
but aren't. It is not a problem with the library design.

> What's all this about necessary re-engineering and rewriting of Phobos? 
> Where the heck did that come from?

You asked what the motivation for Ares was.

>> Why does the C library need replacing? I honestly don't get it.
> Who said it /needed/ replacing? I'm simply talking about applying a 
> small sprinkling of decoupling dust

You write: "If I have, as you say, a system that can't have C's IO 
subsystem, the above will hardly help me at all."

> Forgive me, but that's just printf(). If I have, as you say, a system 
> that can't have C's IO subsystem, the above will hardly help me at all. 
> I suspect you're well aware the C IO subsystem consists of significantly 
> more than just printf? Say, perhaps 50-ish functions? You're going to 
> suggest I stub them all out just because Object.print() calls printf()?

You only need to stub out the functions that pull in the rest. If it's 
only printf, then that's all you need to stub out. If you implement your 
own printf, then it will 'hook' any other calls to printf, and you can 
use it to call your own stuff. It's a common technique - heck, I've done 
it to 'hook' printf to write to a window instead of stdout.

> Given Derek's example:
> 
> void main() {}
> 
> The .map for that is a fine specimen of "unexpected" coupling. It does 
> seem as though much of that is actually in the C library, but then 
> you're arguing most fervently against doing anything to fix any such 
> things.

It is mostly the C runtime library. And I've lived with this for a very 
long time, don't you think I've looked at it to try to reduce the 
minimum required to be pulled in?

> Walter Bright wrote:
>  > Phobos doesn't require floating point support from the processor
>  > unless one actually uses floating point in the application code.
> 
> That turned out to be somewhat less than truthful.

It will be in the next update. I had assumed it was working properly 
when it wasn't - however, printf had nothing to do with the problem. I 
don't know why you call it "futility" - the problem will get fixed. The 
solutions you proposed wouldn't have fixed it.

>  > I also really don't understand why anyone using D would require
>  > not using Phobos. What's the problem?
> 
> With much respect, the 'problem' is perhaps that you don't see any?

Here's the issue I have - postings that adamantly offer solutions 
without identifying the actual problem. Here are some examples:

Solution: remove printf
Alleged problem: printf pulls in floating point formatting code
Actual problem: std.string pulls in floating point formatting code due 
to reference to __fltused. printf does not pull in floating point 
formatting code.
Correct solution: fix compiler to not generate __fltused references in 
library code

Solution: implement separate itoa() for typeinfo
Alleged problem: calling one function in std.string pulls in everything 
in std.string
Actual problem: only a small portion of std.string is actually linked in 
because the free functions are implemented as COMDATs, but due to an 
error in the compiler, lambda functions are not written as COMDATs. This 
causes a reference to std.uni to still be pulled in, pulling in a large 
table in std.uni
Correct solution: fix compiler to generate COMDATs for lambda functions.

I'm glad these two problems came to light, and I'm happy to fix them. 
It's why I ask "What's the problem?" rather than just applying solutions 
without understanding what problem the solution is aimed at. In the two 
cases above, the real problems were found only after I kept asking 
seemingly stupid questions about what problem you were trying to solve 
by removing printf and the call to std.string.toString.

I know you're still bothered by the issue of C's IO being pulled in. I 
have another utility for you - \dm\bin\libunres. Libunres will identify 
any symbols unresolved by a library. Running libunres on phobos.lib gives:

Unresolved externals:
??2 at YAPAXI@Z
??2 at YAPAXIPAX@Z
??3 at YAXPAX@Z
?__stl_throw_length_error at std@@YAXPBD at Z
?__stl_throw_out_of_range at std@@YAXPBD at Z
_ExpandEnvironmentStringsA at 12
_FreeLibrary at 4
_GetFileType at 4
_GetProcAddress at 8
_GetVersion at 0
_IID_IUnknown
_LoadLibraryA at 4
_QueryPerformanceCounter at 4
_RegCloseKey at 4
_RegCreateKeyExA at 36
_RegDeleteKeyA at 8
_RegDeleteValueA at 8
_RegEnumKeyExA at 32
_RegEnumValueA at 32
_RegFlushKey at 4
_RegOpenKeyA at 12
_RegOpenKeyExA at 20
_RegQueryInfoKeyA at 48
_RegQueryValueExA at 24
_RegSetValueExA at 24
_WSACleanup at 0
_WSAGetLastError at 0
_WSAStartup at 8
__Ccmp
__Dmain
__LCMP@
__LDIV@
__U64_LDBL
__ULDIV@
___alloca
___fp_lock
___fp_unlock
___pfloatfmt
__assert
__beginthreadex
__end
__except_list
__fltused
__fputc_nlock
__fputwc_nlock
__global_unwind
__imp__CloseHandle at 4
__imp__CopyFileA at 12
__imp__CopyFileW at 12
__imp__CreateDirectoryA at 8
__imp__CreateDirectoryW at 8
__imp__CreateFileA at 28
__imp__CreateFileMappingA at 24
__imp__CreateFileW at 28
__imp__CreateWindowExA at 48
__imp__DeleteCriticalSection at 4
__imp__DeleteFileA at 4
__imp__DeleteFileW at 4
__imp__DuplicateHandle at 28
__imp__EnterCriticalSection at 4
__imp__FileTimeToSystemTime at 8
__imp__FindClose at 4
__imp__FindFirstFileA at 8
__imp__FindFirstFileW at 8
__imp__FindNextFileA at 8
__imp__FindNextFileW at 8
__imp__FlushViewOfFile at 8
__imp__FormatMessageA at 28
__imp__GetCurrentDirectoryA at 8
__imp__GetCurrentDirectoryW at 8
__imp__GetCurrentProcess at 0
__imp__GetCurrentThread at 0
__imp__GetCurrentThreadId at 0
__imp__GetFileAttributesA at 4
__imp__GetFileAttributesW at 4
__imp__GetFileSize at 8
__imp__GetFullPathNameA at 16
__imp__GetLastError at 0
__imp__GetModuleFileNameA at 12
__imp__GetProcessTimes at 20
__imp__GetSystemInfo at 4
__imp__GetSystemTime at 4
__imp__GetThreadContext at 8
__imp__GetThreadTimes at 20
__imp__GetTickCount at 0
__imp__GetTimeZoneInformation at 4
__imp__GetVersionExA at 4
__imp__InitializeCriticalSection at 4
__imp__InterlockedDecrement at 4
__imp__InterlockedExchange at 8
__imp__InterlockedIncrement at 4
__imp__LeaveCriticalSection at 4
__imp__LocalFree at 4
__imp__MapViewOfFileEx at 24
__imp__MoveFileA at 8
__imp__MoveFileW at 8
__imp__MultiByteToWideChar at 24
__imp__QueryPerformanceCounter at 4
__imp__QueryPerformanceFrequency at 4
__imp__RaiseException at 16
__imp__ReadFile at 20
__imp__RemoveDirectoryA at 4
__imp__RemoveDirectoryW at 4
__imp__ResumeThread at 4
__imp__SetCurrentDirectoryA at 4
__imp__SetCurrentDirectoryW at 4
__imp__SetFilePointer at 16
__imp__SetThreadPriority at 8
__imp__Sleep at 4
__imp__SuspendThread at 4
__imp__UnmapViewOfFile at 4
__imp__VirtualAlloc at 16
__imp__VirtualFree at 12
__imp__WaitForSingleObject at 8
__imp__WideCharToMultiByte at 32
__imp__WriteFile at 20
__imp__lstrcatA at 8
__imp__lstrcmpA at 8
__imp__lstrcpyA at 8
__imp__lstrlenA at 4
__iob
__local_except_handler
__snprintf
__vsnprintf
__xi_a
_accept at 12
_acosl
_asinl
_atan2l
_atanl
_atoi
_bind at 12
_calloc
_cbrtl
_ceill
_closesocket at 4
_connect at 12
_coshl
_erfcl
_erfl
_errno
_execv
_execve
_execvp
_execvpe
_exit
_exp2l
_expl
_expm1l
_fclose
_fdopen
_feof
_fflush
_fgetc
_floorl
_fopen
_fprintf
_fputc
_fread
_free
_fseek
_ftell
_fwide
_fwrite
_gethostbyaddr at 12
_gethostbyname at 4
_gethostname at 8
_getpeername at 12
_getprotobyname at 4
_getprotobynumber at 4
_getservbyname at 8
_getservbyport at 8
_getsockname at 12
_getsockopt at 20
_ilogbl
_inet_addr at 4
_inet_ntoa at 4
_ioctlsocket at 12
_lgammal
_listen at 8
_log10l
_log1pl
_log2l
_logbl
_logl
_malloc
_memchr
_memcmp
_memcpy
_memicmp
_memmove
_memset
_modfl
_nanl
_nearbyintl
_powl
_printf
_realloc
_recv at 16
_recvfrom at 24
_remainderl
_roundl
_select at 20
_send at 16
_sendto at 24
_setsockopt at 20
_shutdown at 8
_sinhl
_socket at 12
_spawnvp
_sprintf
_strcat
_strcmp
_strcpy
_strerror
_strlen
_strncpy
_strrchr
_strtod
_strtof
_strtold
_system
_tanhl
_tgammal
_toupper
_truncl
_ungetc
_vsnprintf
_wcscmp
_wcslen

And that is the *entirety* of what phobos.lib needs for everything in 
phobos. A big chunk of it is Windows API imports. A quick look through 
it shows the following related to C I/O:

_printf
___fp_lock
___fp_unlock
__fputc_nlock
__fputwc_nlock
__iob
_exit
_fclose
_fdopen
_feof
_fflush
_fgetc
_fopen
_fprintf
_fputc
_fread
_free
_fseek
_ftell
_fwide
_fwrite
_ungetc

Applying grep tells us where these are used: modules like std.cstream, 
which is "C" streams, no surprise there. gzio.c, a C function that's 
part of zlib, no surprise there, either. trace's file logging function, 
no big deal, that wouldn't get shipped with a released application. 
std.stdio, no surprise there either, it needs to sync with C's stdio.

There isn't anywhere near 50 routines, and if you don't use cstream, 
zlib, trace, or writef, it's hard to see any difficulty at all. In fact, 
I find it hard to find any other uses of stdio other than printf 
(perhaps I overlooked some?). So why can't 'hooking' printf, as outlined 
above, work?

BTW, internal\dmain2.d also uses printf to print out an error message 
relating to any uncaught exceptions.