D2 and FreeBSD - and: debugging (DWARF); lzma/xz hacks

[Juergen Lock]

On 08/04/2010 22:43, Shin Fujishiro wrote:
> Marco Righele<marco_righele at yahoo.it>  wrote:
>[...]

> I could build dmd 2.047 as follows:
> --------------------
> % mkdir 2.047
> % cd 2.047
> % fetch http://ftp.digitalmars.com/dmd.2.047.zip
> % unzip dmd.2.047.zip
> % cd dmd2/src/dmd/src
> % fetch -o 4191.patch 'http://d.puremagic.com/issues/attachment.cgi?id=629'
> % fetch -o 4198.patch 'http://d.puremagic.com/issues/attachment.cgi?id=632'
> % patch -l -p2<  4191.patch
> % patch -l -p1<  4198.patch
> % gmake -f freebsd.mak
> % cp dmd ~/bin
> --------------------
> Compiler is gcc 4.2.1 on FreeBSD/i386 8-STABLE.  Does this work for you?
>
>
>> When compiling druntime, a static assertion triggers during linking:
>>
>> src/rt/llmath.d(286): Error: static assert  (0x1p+63L == 0x1p+52L) is false
>> gmake: *** [lib/libdruntime.a] Error 1
>
> I'm not sure, but you might be using a non-patched version accidentally.
> Have you copied the newly-built dmd executable to, for example, ~/bin?
>
> I built druntime with the patched dmd 2.047:
> --------------------
> % cd 2.047/dmd2/src/druntime
> % fetch -o 3528.patch 'http://d.puremagic.com/issues/attachment.cgi?id=703'
> % patch -l -p0<  3528.patch
> % gmake -f posix.mak
> --------------------
>
>
>> Finally I was unable to compile phobos, as I encountered errors in
>> different modules, including references to freebsd related files that
>> aren't in the zip I downloaded.
>
> The following file is not in the release zip.  Please download it:
> http://svn.dsource.org/projects/phobos/trunk/phobos/std/c/freebsd/socket.d
>
> And here's a quick-hacked FreeBSD makefile that I use:
> http://gist.github.com/508638
>
> I could build Phobos with the following command lines:
> --------------------
> % cd 2.047/dmd2/src/phobos
> % mkdir std/c/freebsd
> % fetch -o std/c/freebsd/socket.d 'http://svn.dsource.org/projects/phobos/trunk/phobos/std/c/freebsd/socket.d'
> % fetch -o my-freebsd.mak 'http://gist.github.com/raw/508638/b4f85e2de4f41a22a09fb895aa48b7a62fefb71e/my-freebsd.mak'
> % gmake -f my-freebsd.mak
> --------------------
>
>
>> Am I supposed to use a different version of the compiler/libraries ?
>
> The patches should be valid for recent few releases including 2.047.
> I may be wrong though...
> [...]

For the record, here is what I had to change for dmd 2.048:

- 4191.patch and 3528.patch are no longer needed (already applied)

- modfl() was missing in src/druntime/src/core/stdc/math.d only for
   FreeBSD with a comment "// nontrivial conversion", adding it
   back _seems_ to work according to quick tests (was the problem
   that real * i.e. long double * wasn't working properly for C
   bindings in earlier versions and the FreeBSD bindings just hadn't
   been updated yet?)  I've put the patch here:

	http://people.freebsd.org/~nox/tmp/d/dmd.2.048-druntime-modfl.patch

- (at least) std.concurrency and std.exception need to be added to
   my-freebsd.mak, I've put that patch here:

	http://people.freebsd.org/~nox/tmp/d/dmd.2.048-phobos-myfreebsdmak.patch

======

And wrt debugging:  I tested both gdb head (checked out after the
D patches went in) and Doug Rabson's D-aware debugger ngdb which he
annouced as a 'Technology Preview' here:

	http://lists.freebsd.org/pipermail/freebsd-current/2009-August/011071.html

..and I found two things:

a) gdb still chokes on dmd -g debug symbols because the
    D DWARF extensions conflict with DWARF-4:

	http://d.puremagic.com/issues/show_bug.cgi?id=4180

    disabling the DW_TAG_type_unit case in gdb/dwarf2read.c
    at least makes it no longer complain:

	http://people.freebsd.org/~nox/tmp/d/d-gdb-dwarf2read.c.patch

b) both debuggers treat arrays as unsigned long long (like main()'s
    standard string[] args) - at least ngdb prints them correctly
    if I do a manual cast:

	(ngdb) p *cast(char [][] *)&args

    I then looked at the debug symbols using readelf -w and
    found it's actually dmd's fault not the debugger's, the array
    really seems to be marked as the unsigned long long type:

	<1><a4>: Abbrev Number: 3 (DW_TAG_base_type)
	     DW_AT_name        : unsigned long long
	     DW_AT_byte_size   : 8
	     DW_AT_encoding    : 7      (unsigned)
	..
	<2><516>: Abbrev Number: 5 (DW_TAG_formal_parameter)
	     DW_AT_name        : args
	     DW_AT_type        : <a4>
	     DW_AT_location    : 2 byte block: 91 8     (DW_OP_fbreg: 8)

    Any chance this can be changed? :)  (I suspect this is a DWARF-only
    problem i.e. it `works on Windows'?)

======

And finally, I've generated liblzma bindings using bcd, adjusted them
for D2,

	http://people.freebsd.org/~nox/tmp/d/lzma.d

(sorry about the httpd treating .d as application/octet-stream) - and
then made a small threaded .xz compression hack to try out D2,

	http://people.freebsd.org/~nox/tmp/d/xzj.d

  This is very much a WIP but can already be used to compress
files or stdin to another file or stdout (so it can be used in
pipes and with tar's --use-compress-program), using all cores.
Binaries at:

	http://people.freebsd.org/~nox/tmp/d/freebsd8/xzj
	http://people.freebsd.org/~nox/tmp/d/linux/xzj

(both i386 of course)  Run with -h for a small usage message
and see the source for more info.

  Comments are welcome, also if anyone wants to test/port it to
non-FreeBSD/Linux and/or check if the lzma bindings are OK...
Also, should I submit the bindings somewhere?  I checked the
struct sizes and they matched:

	http://people.freebsd.org/~nox/tmp/d/lzmastructsizes-c.c
	http://people.freebsd.org/~nox/tmp/d/lzmastructsizes.d

  These are direct C bindings only tho i.e. I did not make a wrapper
class like std.zlib...

  Ok so that's it for now, cheers,
	Juergen

PS: I've meanwhile found another WIP threaded .xz compressor written
in C, but it is specific to tarfiles (it even indexes them and can
extract a single file quickly), and it uses a private branch(?)
of liblzma:

	http://github.com/vasi/pixz
	http://github.com/vinzenz/liblzma