Proposed Phobos equivalent of wcswidth()

[H. S. Teoh]

On Wed, Jan 17, 2018 at 05:06:05AM +0000, Dmitry Olshansky via Digitalmars-d wrote:
> On Tuesday, 16 January 2018 at 23:01:19 UTC, H. S. Teoh wrote:
[...]
> > One thing, though: I think it would benefit us all if we could
> > import at least gen_uni into Phobos, so that in the future when we
> > need to update std.uni to a new version of Unicode, it can be
> > (mostly) automated.  It's better to have the tools to generate the
> > tables in Phobos itself, than to be dependent on an external repo
> > that may go out-of-sync eventually.
> 
> Yes but it’s non-trivial at the moment, if you take a look at script
> to generate stuff it takes both 32-bit and 64-bit executables to
> populate tables.
> 
> I think having it in tools repo should be fine though. Last time I
> tried to update to Unicode 10, I found one table in Phobos that is
> missing from generator (ooops!).

I took a first stab at integrating this into dlang/tools:

	https://github.com/quickfur/tools/tree/unicode_gen

So far, I can get the 64-bit generator to run and produce the generated
unicode_*.d files. Unfortunately they are missing the 32-bit data,
because I couldn't get a 32-bit dmd toolchain working on my PC.

Maybe you could take a look and submit PRs against that branch for any
fixes you'd like to get in?  I'll see if I can somehow get 32-bit
working on my PC.

Alternatively, maybe the solution is to hack the Trie code so that it
uses explicit int sizes rather than size_t, then we can use it to
generate both 32-bit and 64-bit tables without requiring the host
platform to support both.  I imagine we may have problems getting the
tools repo to build on the autotester once we integrate gen_uni into the
makefile, unless we do something like this.


> > When I get around to making a PR for strwidth AKA displayWidth, the
> > plan is to check-in compileWidth.d in some form into Phobos
> > somewhere, so that somebody else can pick it up and improve the
> > implementation in the future if I'm not around / unavailable.
> > 
> > If we can get gen_uni into Phobos, perhaps we can even include the
> > displayWidth table generation in gen_uni too, so that all the table
> > generation code is in one place.
> 
> Right. A good step would be to move it to tools, then add your code.
[...]

Good idea.  Well, I started with the branch linked above in my fork of
dlang/tools.  If I can get it off the ground, I'll add the displayWidth
stuff in as well, then formulate a PR to add displayWidth to std.uni.

Well, technically I don't need to wait for that, since I could just add
the precomputed table directly into std/internal/unicode_tables.d. But
it's probably better to let the generator do the job instead.  A
precomputed table is rather hard to review for correctness when it comes
PR review time. :-D


T

-- 
Don't get stuck in a closet---wear yourself out.