[RFC] ColorD

[Jens Mueller]

Chad J wrote:
> On 10/21/2012 06:55 PM, Jens Mueller wrote:
> >Chad J wrote:
> >>On 10/21/2012 06:11 PM, Jens Mueller wrote:
> >>>Chad J wrote:
> >>>>On 10/21/2012 05:01 PM, Jens Mueller wrote:
> >>>>
> >>>>It seems to have a hard ncurses/termcap/etc dependency.
> >>>
> >>>Yes. I think you cannot make it portable without. Please proof me wrong
> >>>and I'll fix this.
> >>>
> >>
> >>Well, traditionally it's done with automake/autoconf.  You'd end up
> >>with preprocessor defines that tell you whether the lib has been
> >>statically linked or not.  This isn't available here because Phobos
> >>doesn't use these as a build system and I hope it never does.
> >
> >I mean to detect if your terminal is ANSI compatible without adding
> >another dependency.
> >It's easy to provide different version(...) to support different modes.
> >One could do something like:
> >1. Check at run time if ncurses etc. are available.
> >   * If they are use them.
> >   * Otherwise fall back to ANSI codes or throw an Exception.
> >
> >What do you think?
> >
> 
> I completely agree.
> 
> The difficulty I encountered is actually /doing/ the runtime
> detection.  Does Phobos have a way to dynamically link .so files
> yet?

It is possible since very long time to link dynamically using dmd.

> If yes, then we could search the obvious places
> ("/lib/libncurses.so.5" matches on my machine right now) and link to
> any files found.

For that purpose in particular I have written ddl.
https://github.com/jkm/ddl
http://jkm.github.com/ddl/ddl.html
Just haven't found the time to integrate with the terminal stuff.

> Since color is probably never necessary for program correctness, I
> think it is acceptable to ignore color formatting and produce plain
> text when detection fails.  It would make sense to make this
> configurable though: the Terminal object could have a
> .throwOnDetectionFailure flag that can be set if people want to be a
> bit more hardcore about it.

Sounds useful.

> >>>>Ultimately I expect it to work with writeln or writefln to make it
> >>>>discoverable and easy to work with.
> >>>
> >>>One could try this. At least for Linux. You just have to add the
> >>>appropriate escape sequences. But this won't work on Windows.
> >>>
> >>
> >>I remember having a plan for this.  See below.
> >>
> >>>>Back then I did design a format spec for introducing colors into
> >>>>format strings:
> >>>>www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html
> >>>
> >>>I doubt that the Phobos maintainers will accept this. This is very
> >>>invasive.
> >>
> >>Hmmm, depends what is meant by invasive.
> >>
> >>I feel it's the only way to have discoverable and concise syntax.
> >>I'd be pretty disappointed if they didn't, regardless of who submits
> >>the pull request.
> >>
> >>I remember it being possible in Phobos to determine the destination
> >>of the format operation.  If the destination is a string in memory,
> >>then no color formatting would be applied.  If the destination is a
> >>Linux terminal of some kind, then some ncurses terminal info would
> >>be looked up (possible a cached lookup) and escape sequences
> >>generated based on that.  If the destination is a Windows terminal,
> >>then these approaches can be considered:
> >>(1) Split the formatted text up on the color format boundaries.
> >>Send the slices into the stream one by one, calling the necessary
> >>WinAPI color formatting functions inbetween.  I think this might not
> >>have been possible with Phobos' architecture.
> >>(2) Insert ANSI escape sequences into the text.  The I/O code for
> >>Windows would then have to intercept these and convert them into the
> >>appropriate WinAPI calls.  I think this was possible, and even
> >>distinguishable from the case of writing to a string in memory.
> >>
> >>If the invasiveness worry comes from the possibility of dumping
> >>escape sequences into non-terminal destinations, then I hope the
> >>above wall of text can alleviate that concern.
> >
> >Checking whether something is a terminal can be done using isatty on the
> >file handle. I think this will work.
> >But it is invasive because you want to add it to the formatting spec. Is
> >this the usual way it is done? I don't know how it is done in Python
> >or other languages.
> >
> 
> Is it relevant?  I posit that having format syntax is simply better
> than not.  This is on an objective basis.

Maybe. Don't know. But in any case this is a add on. On top of the basic
API. It is a Phobos integration thing.

> There is no weakness to this.  The only shred of a counterargument I
> can think of is that it makes the format strings more difficult to
> learn. Other than that, it is possible to detect the destination of
> the formatter, so color codes will never end up in places where they
> shouldn't.  A conservative approach to this should handle most
> desires and never interfere with all the people with no interest in
> color.
> 
> On the upshot are the things I've mentioned:
> - A format specifier is potentially more discoverable.
> - A format specifier is more concise.  This keeps your lines from
> wrapping.  They are probably too long already.

Do you consider this
writecf(Color.red, "something %s", "here")
concise as well?

> - To cement the previous point: nesting requires a few extra
> characters with a format specifier, rather than a couple extra
> /lines/ for extra function calls.

Don't understand this point. Can you give an example?

> - Calls to stateful console functions allow people to write bugs
> like saving console state and then forgetting to restore it (or
> throwing an exception and neglecting to restore from within a scope
> guard).  Format specifiers do not have this problem.

The same holds for
writecf(Color.red, "something %s", "here")

> - etc (I'm sure I'm forgetting one or two.)
> 
> These are the reasons why my ideal language has color formatting
> built into its I/O routines.

Just wanted to point out that instead of that you can add writec*
functions. I think the only thing is that these are less discoverable
but they also work without formatting, e.g.
writec(Color.red, "my text");

> >>I run a Gentoo system where things are configured to use color
> >>output wherever possible.  The portage devs went through all of the
> >>necessary contortions to get Python to output colored text, somehow.
> >>I feel the end result is indispensable.  Color is an extremely
> >>useful tool for making sure that the user doesn't overlook important
> >>bits while scanning text.  Outside of Gentoo, I find this most
> >>notable in grep: uncolored grep output is just awful, but the
> >>coloring makes it possible to easily identify why the regular
> >>expression behaved the way it did.
> >>
> >>I look forward to a better CLI ecosystem where highly reliable D
> >>programs are written quickly and write beautiful colored output ;)
> >
> >It is. I'd like you to join and get this done.
> >And you're right we should have Phobos integration in mind. Maybe they
> >will add it. But in the mean time we can have a separate module.
> >
> >Jens
> 
> I think this is very reasonable.  Sounds like a plan!
> 
> I'll probably make pull requests when I allocate time to work on this.

That would be nice.
Let's coordinate with Robik. I think having Robik's ANSI stuff as a fall
back would be nice. And we need to decide on a repository.

> Would implementing the format spec make sense?  It might give me a
> good excuse to write an snprintf alternative in D that can be used
> at compile-time and won't introduce nasty link-time dependencies.

This is probably interesting for Phobos. But I'm not the one to make a
decision. The core Phobos developers should decide.
Hopefully somebody is reading this.

Jens