[Issue 22960] New: importC: K&R-style functions assume variadic calling convention

[d-bugmail at puremagic.com]

https://issues.dlang.org/show_bug.cgi?id=22960

          Issue ID: 22960
           Summary: importC: K&R-style functions assume variadic calling
                    convention
           Product: D
           Version: D2
          Hardware: x86_64
                OS: Linux
            Status: NEW
          Keywords: ImportC
          Severity: normal
          Priority: P1
         Component: dmd
          Assignee: nobody at puremagic.com
          Reporter: duser at neet.fi
                CC: duser at neet.fi

on 64-bit linux/posix:

// file1.c
void other(int);
long return_arg1(long x) { return x; }
int main()
{
        return_arg1(-1); // put 0xff in AL
        other(0);
        return 0;
}

// file2.c
void other(x)
        int x;
{
        // never reached
}

compile using "dmd file1.c file2.c", run to observe segfault

it crashes because of the variadic function prologue in other():

https://en.wikipedia.org/wiki/X86_calling_conventions#System_V_AMD64_ABI

> If the callee is a variadic function, then the number of floating point arguments passed to the function in vector registers must be provided by the caller in the AL register.

it's called through a non-variadic prototype and the body is in a different
file so dmd doesn't know AL has to be cleared first

the segfault is because the variadic code in other() does a jump depending on
the value of AL to only save the used registers, which fails if AL contains a
garbage value other than 0-8

the zlib library has function bodies in K&R style and assumes that this works
if __STDC_VERSION__ is defined (function prototypes in headers will contain the
parameters instead of () in that case). the zlib bindings in phobos also don't
use variadic functions so they wouldn't work with a dmd-compiled zlib because
of this

--