[Dlang-internal] Potential changes to DDoc

[Sebastian Wilzbach]

David, thanks a lot for championing this project and pushing it forward!
I saw that you already joined Slack and #dmd, so please feel free to ask 
questions there.

There are two advises I can give you that as they will save you a lot of 
frustration and work:

> Open your PR early

Most discussions happen either on GitHub or Slack.
As not everyone is active on Slack, a GitHub PR is usually a good place 
for a in-depth conversation.
Also looking at your work allows us to give you feedback early on and 
avoid unnecessary work / it drifting in the wrong direction.

> Split your changes into minimal chunks

Ideally each point on your TODO list translates to one small PR.
This allows fast review and as a bonus avoids rebasing conflicts.
This point is usually underestimated, but it's vastly easier to pull a 
30 lines change (+tests) than a 1K diff.

On 2018-01-20 19:33, David Gileadi via Dlang-internal wrote:
> (I don't know if this is the right forum for this post, so if not
> please let me know.)
> 
> I'm working on adding Markdown support to DDoc. I'm doing it by
> modifying the highlightText method in doc.d [1].
> 
> I've been trying to make my changes non-breaking, in that existing
> DDoc files should be rendered the same as they currently are. However
> I've realized that there may need to be some breakage, and I'd like to
> discuss it.
> 
> First of all, the CSS styles in the default DDoc theme remove all
> styling from headers, blockquotes, list items, pre and code sections,
> etc. [2] The effect of this is that Markdown features like headers,
> quotes, etc. don't display correctly under the default theme. I'd like
> to modify the default DDoc theme to avoid de-styling these elements.
> 
> Secondly, HTML is treated differently under Markdown and DDoc. In DDoc
> anything directly within an HTML tag (i.e. between < and >) is passed
> through unchanged. However if it's not part of the tag it is treated
> as regular DDoc text and subject to regular processing. In Markdown
> all text between a given set of HTML tags is passed through unchanged
> [3]. I'd prefer to switch to the CommonMark handling, allowing whole
> blocks of HTML to be passed through directly.
> 
> Thirdly and most controversially is how DDoc handles blank lines
> between paragraphs. Markdown wraps paragraphs in <p></p> tags and
> collapses multiple blank lines between paragraphs (i.e. ten blank
> lines is treated the same as one). DDoc inserts $(DDOC_BLANKLINE)
> macros on each blank line it encounters, which in the default theme
> inserts <br><br>, in the Phobos theme inserts <div
> class="blankline"/>, and for LaTeX inserts /par. Phobos docs also have
> the common practice of hand-wrapping paragraphs in $(P) macros which
> inserts <p></p> tags.
> 
> The result of DDoc's handling is that under the default theme if you
> have, say, three blank lines between paragraphs then it results in
> five blank lines being rendered between them.
> 
> I'm not yet sure it's feasible, but I'd really like to change DDoc to
> treat paragraphs the way Markdown does--wrap them in, say, a $(P)
> macro and collapse multiple blank lines. Any desired extra blank lines
> could be inserted by hand, via the $(BR) macro or similar. However
> this could change the rendering of existing documentation which
> depends on the $(DDOC_BLANKLINE) macro.
> 
> What do you think about the proposed changes? Should I try to
> implement any or all of them?
> 
> -Dave
> 
> 
> Appendix A: Planned Markdown support
> ------------------------------------
> 
> I plan to support the following Markdown features, as specified by
> CommonMark [4]:
> 
> - Headings, both the ATX and setext kind
> - Styling text with * (but not with _)
> - Quoting text blocks with >
> - Code blocks fenced with ```, ~~~ and DDoc's existing ---
> - Backtick-quoted code as it currently works in DDoc
> - Nesting lists
> - Thematic breaks (<hr>)
> - Inline links, although I'm not sure if I will support reference links 
> [5]
> - Markdown escapes with \
> - HTML, although that's a discussion item (see above)
> 
> I don't plan to support the following Markdown features:
> 
> - Indented code blocks because existing DDoc comments are often already 
> indented
> - Using --- for HRs and second-level headings because DDoc uses it as
> a code block fence
> - Tables
> - Backtick-quoting code across multiple lines
> - Github-specific features like task lists, mentioning people and
> teams, referencing issues and pull requests, and emoji
> 
> 
> [1] https://github.com/dgileadi/dmd/tree/ddoc-markdown
> [2] 
> https://github.com/dlang/dmd/blob/master/res/default_ddoc_theme.ddoc#L70
> [3] http://spec.commonmark.org/0.28/#html-blocks
> [4] http://spec.commonmark.org/0.28/
> [5] http://spec.commonmark.org/0.28/#reference-link