Update roll-up on my arsd libraries

Thu Apr 13 09:04:18 PDT 2017

I haven't posted in announce for a while, so I figured I'd give 
you guys an update on what's been happening in my libs. This 
message is a big long, so feel free to just skim to see if you 
are interested in anything.

Here's my three main public repos:

https://github.com/adamdruppe/arsd
https://github.com/adamdruppe/terminal-emulator
https://github.com/adamdruppe/adrdox

  * * * * *

Going bottom to top: adrdox is the documentation generator for my 
http://dpldocs.info/ website. In the last few months, it:

* Is easier to use by third parties. If you download it and hit 
`make`, then run it from its own source directory (that's 
important right now) `./doc2 path/to/your/project`, it will 
generate your docs. So you don't have to be on my computer to run 
it anymore!

* It has a javascript search option, so you can use it 
out-of-the-box with search enabled on a static site, like  github 
pages.

* It has gained a lot of automatic cross referencing in source 
code.

* The documentation generating language got a few more features 
like footnotes. See the syntax doc here: 
http://dpldocs.info/experimental-docs/adrdox.syntax.html

  * * * * *

terminal-emulator is my custom Linux terminal emulator. It 
includes four programs:

* a Linux program that works like xterm, just with my preferences 
and features. For example, it adjusts the palette so if a program 
prints yellow on white, it will adjust to a darker yellow to make 
it easier to read.

* a Windows program that works just like Linux, but instead of 
running a local shell, it ssh connects to a Linux box and runs 
the shell there.

* a Linux program that enables tabbing and detaching of terminal 
sessions, similar to GNU Screen, but forwarding my custom 
features. For example, scrollback with shift+pageup works through 
all levels of nesting, and it can do bracketed pastes through the 
levels.

* and a nested terminal emulator that I rarely use, but could be 
used to split screens and such.

It also holds the stand-alone terminalemulator.d core, which is 
independent of any UI.

It has mostly gotten bug fixes over the last several months, 
including a major one on scrolling back long lines. I use it for 
all my terminal emulation need (which is almost everything I do 
on computers!), so the more I use it, the more fixes it gets.

  * * * * *

arsd contains my generally useful library modules. See the docs 
here: http://dpldocs.info/experimental-docs/arsd.html

Some of the modules are obsolete or experimental, but many of 
them are stable for many years, including:

* cgi.d can be used for D web programs using existing libraries - 
it uses simple process and thread model rather than the 
newfangled async event loops, which means it is compatible with 
almost any code you write. It works with cgi, fastcgi, scgi, or 
you can run an embedded http server in it. It stands alone, 
though web.d (I haven't used it for a while so it might need some 
updating) and others can build on top of it.

It hasn't changed for a while, no new updates here.

* dom.d is a html or xml dom parser, capable of reading tag soup. 
Its API is similar to Javascript, so if you have used the browser 
apis before, you should be able to dive in pretty quickly. It has 
no required dependencies.

In recent months, it has had a few small bug fixes and gained a 
new api: `optionSelector` 
http://dpldocs.info/experimental-docs/arsd.dom.Element.optionSelector.html

It uses the mostly-CSS3-compatible selector syntax, same as 
querySelector and requireSelector, but returns a null-safe 
object, meaning you can use it for scrapes.

optionSelector(".foo").innerText; // instead of throwing or 
segfaulting if .foo does not exist, it just returns an empty 
string

It has also gained a Document.fromUrl function which will 
download and parse some web site. This requires 
characterencodings.d and http2.d as well.

dom.d has also received a few more bug fixes in reading tag soup 
as I use it for more and more web scraping. See also: 
htmltotext.d which uses dom.d to convert html to plain text and 
html.d for more html helper functions. It includes a css expand 
thing similar to scss. It hasn't changed much recently, either.

* database.d, postgres.d, sqlite.d, mysql.d, and mssql.d are 
wrappers of the C libraries. They haven't changed much recently.

* http2.d is a http/1.1 client library. (It is the successor to 
my http.d lib and does not yet have http/2 support. And maybe 
never will since 1.1 client is plenty good enough for everything 
I use it for).

It has gotten some new stuff recently:

   - HttpClient is a class that works kinda like a browser, with 
cookie storage, connection reuse, and relative link handling. It 
has received a number of bug fixes in the last few months, 
especially with regard to SSL (for which it requires OpenSSL 
library btw).

   - The HttpApiClient template has convenience methods for 
working with Oauth2 json apis. It is still a work in progress but 
you can read some about it here: 
http://dpldocs.info/experimental-docs/arsd.http2.HttpApiClient.html It requires jsvar.d

* jsvar.d and script.d provide a Javascript-esque type for D with 
easy interop into the rest of the language. jsvar itself just 
provides the data type, and script.d expands it with a dynamic 
scripting language interpreter too (no outside dependencies btw). 
They've changed a bit recently:

   - jsvar is even more loosely typed than it was before: 
accessing a member of a null object simply returns null too. This 
is convenient for accessing nested JSON objects where they may 
not be present (similar to the use case for optionSelector in 
dom.d), though makes errors harder to find. I'll probably go back 
to some kind of middle ground on that eventually.

   - script has more bug fixes and gains from every jsvar change. 
Notably, it can now more reliably wrap native D class objects, 
but it does so in an opaque way, just holding and returning the 
object on demand, unless the object's specific methods are marked 
with a UDA `@"arsd_jsvar_compatible"`. That UDA has nicer aliases 
in a few of my modules like `@scriptable`. The reason it is a 
string though is I can mark it in other modules without 
introducing a dependency on jsvar so the user doesn't pay for it 
if they don't use it.

   - The script language now allows identifiers-with-dashes, since 
I like how they look in scripts a lot. To do subtraction, you 
must put spaces around the - character.

* terminal.d is a terminal client library, similar to ncurses. It 
allows you to move the cursor around the terminal, go to full 
screen mode, output color, input all keys and mouse events.

In recent months, it has gained 24-bit color output support (with 
automatic approximation to lower color fallback), though I 
generally suggest you keep to 8 color output for best results in 
terminal programs.

It also got a `getline` function with user editing and 
autocompletion available, unified some of the key input events, 
and a ScrollbackBuffer which is clunky to use, but allows you to 
have individual scroll buffers scattered around the screen.

It also received a few small bug fixes for Mac OS X.

* simpleaudio, joystick, and gamehelpers haven't been changed 
yet, but it is my plan to almost completely redo them. I am 
reasonably happy with joystick.d, but simpleaudio.d is probably 
going to be gutted and rewritten. If you actually use it as-is, 
let me know and I'll work with you on compatibility, but I doubt 
that...

* simpledisplay.d has gotten a LOT of work lately.

   - Its ScreenPainter is now more reliable if you pass it to 
other functions. It does refcounting but also sets the impl 
values back in dtors so it acts like a hybrid value/reference 
type.

   - The NotificationAreaIcon now works on Windows and Linux, with 
balloon notifications.

   - There's a new postEvent API with collapsing and timeouts, 
allowing multithreaded programs to run efficiently.

   - It has support for creating OpenGL 3+ contexts.

   - There are more exposed low level events - simpledisplay aims 
to be simple for many things, but not get in your way if you want 
to do more with it than it was specifically designed for.

   - It has timer and Linux file read support in its event loop 
now, though postEvent can be better than Timer for game purposes, 
and the Linux file read is use-at-your-own-risk experimental.

   - ScreenPainter can do clipping and transformations.

   - Sprite can now be drawn upon, just like SimpleWindow. Sprite 
is a class that contains a display-side image for quick blitting 
(kinda like a 2d version of a opengl texture).

   - SimpleWindow can construct more window types like popup 
menus, for use building custom UIs, among other things.

   - OperatingSystemFont gives a way to load a font off the OS and 
draw with it in a ScreenPainter.

   - You can captureInput to a window, getting mouse tracking or 
even confinement.

   - There are platform-specific functions for handling global 
hotkeys. Still work in progress in unifying the APIs.

   - simpledisplay.d can now survive an X server disconnect, and 
can even migrate windows from one X server to another! Think GNU 
Screen for GUI apps.

   - The documentation has been expanded, though it still needs 
cleanup, it includes more and more examples. See: 
http://dpldocs.info/experimental-docs/arsd.simpledisplay.html

   - Other little features and bug fixes.

I haven't used SDL or anything similar for years thanks to 
simpledisplay.d. Its only dependency is color.d, leading to 
simple builds.

* And the huge one in the last month, minigui.d is a small GUI 
widget library that aims to use native widgets on Windows and a 
small set of custom ones on Linux (in the future, it might also 
support GTK widgets, though I don't plan to write that myself, 
there is just another user who said he might). It has no 
dependencies except for simpledisplay.d and color.d and the 
operating system - no complicated library build process, no huge 
dlls to distribute with your program.

http://dpldocs.info/experimental-docs/arsd.minigui.html

Before now, it was fairly usable on Windows, but only had buttons 
on Linux. Now, it is about equally featured on both with widgets 
like scroll areas, checkboxes, buttons, progress bars, menus, 
drop down selects, textareas, and more. It aims to cover the 
basic boxes that HTTP 4 forms did, while also being easy to 
define your own widgets - you just subclass it and draw in the 
area it assigns you. It now even has an OpenGL widget.

It automatically sizes and lays out your widgets and has an event 
system similar to Javascript. With add on modules like 
minigui_xml, you can create windows from XML definitions, and it 
is script.d compatible. When I'm ready, I'll post that too, and 
hopefully you can take basic web knowledge over to desktop 
land... without just embedding browsers and calling them desktop 
apps. My largest program is about 800 KB, and that's all it needs.

Since it is built on simpledisplay, it also gets X connection 
disconnect survival or migration!

Old screenshot: http://arsdnet.net/guitest.png
New screenshot: http://arsdnet.net/minigui.windows.new.png
New screenshot, Linux version: 
http://arsdnet.net/minigui.linux.d.png
an app I'm writing with it: http://arsdnet.net/minigui-sprite.png
xml app screenshot: http://arsdnet.net/minigui-scripted-app.png

Demo program source code: http://arsdnet.net/guitest.d

I'll write more about minigui.d and do a separate announcement 
for it when it reaches version 1.0, which will probably be in 
about another month.

The look on Linux is based on Windows 95, I think it is elegant 
and usable, and is also very efficient for remote X connections, 
so I can use them from my laptop on slow wifi.

minigui also has more add on widgets: 
https://github.com/adamdruppe/arsd/tree/master/minigui_addons

In the core is a file picker, which is very fast on Linux and 
native on Windows. In addons, we add a color picker and a 
terminal emulator widget (depends on the terminal emulator core, 
mentioned above in this message).

Look forward to more on this, it is close to being ready for 
general use, and then we'll have another chapter in the D gui 
ecosystem. dlangui <https://github.com/buggins/dlangui>, I hope 
you like some friendly competition :)