[RFC] ColorD

[Chad J]

On 10/21/2012 06:55 PM, Jens Mueller wrote:
> Chad J wrote:
>> On 10/21/2012 06:11 PM, Jens Mueller wrote:
>>> Chad J wrote:
>>>> On 10/21/2012 05:01 PM, Jens Mueller wrote:
>>>>
>>>> It seems to have a hard ncurses/termcap/etc dependency.
>>>
>>> Yes. I think you cannot make it portable without. Please proof me wrong
>>> and I'll fix this.
>>>
>>
>> Well, traditionally it's done with automake/autoconf.  You'd end up
>> with preprocessor defines that tell you whether the lib has been
>> statically linked or not.  This isn't available here because Phobos
>> doesn't use these as a build system and I hope it never does.
>
> I mean to detect if your terminal is ANSI compatible without adding
> another dependency.
> It's easy to provide different version(...) to support different modes.
> One could do something like:
> 1. Check at run time if ncurses etc. are available.
>    * If they are use them.
>    * Otherwise fall back to ANSI codes or throw an Exception.
>
> What do you think?
>

I completely agree.

The difficulty I encountered is actually /doing/ the runtime detection. 
  Does Phobos have a way to dynamically link .so files yet?

If yes, then we could search the obvious places ("/lib/libncurses.so.5" 
matches on my machine right now) and link to any files found.

Since color is probably never necessary for program correctness, I think 
it is acceptable to ignore color formatting and produce plain text when 
detection fails.  It would make sense to make this configurable though: 
the Terminal object could have a .throwOnDetectionFailure flag that can 
be set if people want to be a bit more hardcore about it.

>>>> I'll admit when I started trying to work on doing this thing, I
>>>> never got anything onto the screen.  What stopped me was that I
>>>> couldn't figure out how to detect ncurses/termcap/etc.  I was going
>>>> to shoot for Phobos inclusion and making Phobos always link with
>>>> ncurses seems like a bad idea.
>>>
>>> Dependence on Phobos is bad. If you can detect whether a terminal is
>>> ANSI compatible then this mode should be default. But I don't know how
>>> to detect this.
>>>
>>
>> Wrong direction on the dependency.  I wouldn't expect Terminal
>> coloring/detection to rely on Phobos.  I'd expect it to be one of
>> the lower-level modules built into Phobos.
>
> I mean it's bad to have Phobos depend on ncurses.
> Though one can go with loading at run time.
>

Yes.

>>>> Ultimately I expect it to work with writeln or writefln to make it
>>>> discoverable and easy to work with.
>>>
>>> One could try this. At least for Linux. You just have to add the
>>> appropriate escape sequences. But this won't work on Windows.
>>>
>>
>> I remember having a plan for this.  See below.
>>
>>>> Back then I did design a format spec for introducing colors into
>>>> format strings:
>>>> www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html
>>>
>>> I doubt that the Phobos maintainers will accept this. This is very
>>> invasive.
>>
>> Hmmm, depends what is meant by invasive.
>>
>> I feel it's the only way to have discoverable and concise syntax.
>> I'd be pretty disappointed if they didn't, regardless of who submits
>> the pull request.
>>
>> I remember it being possible in Phobos to determine the destination
>> of the format operation.  If the destination is a string in memory,
>> then no color formatting would be applied.  If the destination is a
>> Linux terminal of some kind, then some ncurses terminal info would
>> be looked up (possible a cached lookup) and escape sequences
>> generated based on that.  If the destination is a Windows terminal,
>> then these approaches can be considered:
>> (1) Split the formatted text up on the color format boundaries.
>> Send the slices into the stream one by one, calling the necessary
>> WinAPI color formatting functions inbetween.  I think this might not
>> have been possible with Phobos' architecture.
>> (2) Insert ANSI escape sequences into the text.  The I/O code for
>> Windows would then have to intercept these and convert them into the
>> appropriate WinAPI calls.  I think this was possible, and even
>> distinguishable from the case of writing to a string in memory.
>>
>> If the invasiveness worry comes from the possibility of dumping
>> escape sequences into non-terminal destinations, then I hope the
>> above wall of text can alleviate that concern.
>
> Checking whether something is a terminal can be done using isatty on the
> file handle. I think this will work.
> But it is invasive because you want to add it to the formatting spec. Is
> this the usual way it is done? I don't know how it is done in Python
> or other languages.
>

Is it relevant?  I posit that having format syntax is simply better than 
not.  This is on an objective basis.

There is no weakness to this.  The only shred of a counterargument I can 
think of is that it makes the format strings more difficult to learn. 
Other than that, it is possible to detect the destination of the 
formatter, so color codes will never end up in places where they 
shouldn't.  A conservative approach to this should handle most desires 
and never interfere with all the people with no interest in color.

On the upshot are the things I've mentioned:
- A format specifier is potentially more discoverable.
- A format specifier is more concise.  This keeps your lines from 
wrapping.  They are probably too long already.
- To cement the previous point: nesting requires a few extra characters 
with a format specifier, rather than a couple extra /lines/ for extra 
function calls.
- Calls to stateful console functions allow people to write bugs like 
saving console state and then forgetting to restore it (or throwing an 
exception and neglecting to restore from within a scope guard).  Format 
specifiers do not have this problem.
- etc (I'm sure I'm forgetting one or two.)

These are the reasons why my ideal language has color formatting built 
into its I/O routines.

>>> I added writecf, writec, etc. with additional arguments.
>>> writec(Color.red, "some text")
>>> or
>>> writecf(Color.red, "%s", "some text")
>>> This is fine I think. But better options may be worth investigating.
>>>
>>> Jens
>>
>> I really think this should be in Phobos.  If it doesn't go into
>> Phobos, then people will write crappy terminal apps with no color.
>> If it does go into Phobos, then the better devs will see the
>> opportunity and use it.  Something 3rd party is much less
>> discoverable and won't have nearly as much impact.  The use case is
>> almost all CLI apps, so it's not like an uncommon corner-case or
>> something.
>
> True. When I asked there was less/no interest. I think we should just
> join and make one module that shows up top ten when googled for "D
> terminal/console color".
>
>> I run a Gentoo system where things are configured to use color
>> output wherever possible.  The portage devs went through all of the
>> necessary contortions to get Python to output colored text, somehow.
>> I feel the end result is indispensable.  Color is an extremely
>> useful tool for making sure that the user doesn't overlook important
>> bits while scanning text.  Outside of Gentoo, I find this most
>> notable in grep: uncolored grep output is just awful, but the
>> coloring makes it possible to easily identify why the regular
>> expression behaved the way it did.
>>
>> I look forward to a better CLI ecosystem where highly reliable D
>> programs are written quickly and write beautiful colored output ;)
>
> It is. I'd like you to join and get this done.
> And you're right we should have Phobos integration in mind. Maybe they
> will add it. But in the mean time we can have a separate module.
>
> Jens

I think this is very reasonable.  Sounds like a plan!

I'll probably make pull requests when I allocate time to work on this.

Would implementing the format spec make sense?  It might give me a good 
excuse to write an snprintf alternative in D that can be used at 
compile-time and won't introduce nasty link-time dependencies.