Showing unittest in documentation (Was Re: std.unittests [updated] for review)

[Andrei Alexandrescu]

On 1/24/11 2:37 PM, Steven Schveighoffer wrote:
> On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu
> <SeeWebsiteForEmail at erdani.org> wrote:
>
>> On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:
>>> On 1/24/11 1:50 PM, Jens Mueller wrote:
>>>> Jonathan M Davis wrote:
>>>>> I think that it's been discussed a time or two, but nothing has been
>>>>> done about
>>>>> it. It wouldn't be entirely straightforward to do. Essentially,
>>>>> either a
>>>>> unittest block would have to be generated from the Examples section
>>>>> in the
>>>>> documentation, or you'd have to have some way to indicate that a
>>>>> particular
>>>>> unittest block got put into the documentation as an Examples section.
>>>>> It's
>>>>> certainly true that it would be ideal to have a way to avoid the
>>>>> duplication,
>>>>> but we don't have one at the moment, and it hasn't yet been a high
>>>>> enough
>>>>> priority to sort out how to do it and implement it.
>>>>
>>>> I see. I understand that it does not have high priority. Just wondered
>>>> whether ...
>>>>
>>>> Jens
>>>
>>> The change is much simpler than what Jonathan suggests. A change can be
>>> made such that any unittest preceded by a documentation comment is
>>> automatically considered an example.
>>>
>>> /**
>>> Example:
>>> */
>>> unittest
>>> {
>>> writeln("This is how it works.");
>>> }
>>>
>>>
>>> Andrei
>>
>> BTW I consider this a very important topic. We have _plenty_ of
>> examples that don't work and are not mechanically verifiable. The
>> reasons range from minor typos to language changes to implementation
>> limitations. Generally this is what they call "documentation rot".
>> This is terrible PR for the language.
>>
>> Changing ddoc to recognize documentation unittests would fix this
>> matter once and forever.
>>
>> Last but not least, the "----" separators for code samples are awful
>> because no editor recognizes them for anything - they confuse the hell
>> out of Emacs for one thing.
>
> This only makes sense if:
>
> 1. The unit test immediately follows the item being documented
> 2. The unit test *only* tests that item.

That's the what current examples do for virtually all of Phobos.

> The second one could be pretty annoying. Consider cases where several
> functions interact (I've seen this many times on Microsoft's
> Documentation), and it makes sense to make one example that covers all
> of them. Having them 'testable' means creating several identical unit
> tests.
>
> One way to easily fix this is to allow an additional parameter to the
> comment:
>
> /**
> Example(Foo.foo(int), Foo.bar(int)):
> */
> unittest
> {
> auto foo = new Foo;
> foo.foo(5);
> foo.bar(6);
> assert(foo.toString() == "bazunga!");
> }
>
> The above means, copy the example to both Foo.foo(int) and Foo.bar(int)

Why would I force the reader to read the same example twice? And why 
would I run the same unittest twice?

> An alternative that is more verbose, but probably more understandable:
>
> /**
> Example:
> Covers Foo.foo(int)
> Covers Foo.bar(int)
> */
>
> Of course, a lack of target just means it applies to the item just
> documented.

I find documented unittests attractive mainly because they're _simple_. 
As soon as we start to add that kind of stuff... exponential decay.

> One other thing, using writefln is considered bad form in unit tests
> (you want *no* output if the unit test works). But many examples might
> want to demonstrate how e.g. an object interacts with writefln. Any
> suggestions? The assert line above is not very pretty for example...

Yah, that is an issue. For examples that do non-unittesty stuff (e.g. 
writeln, use sockets etc.) we can still use the old-style documentation. 
By the way, for all examples that don't explicitly describe writeln, we 
shouldn't use writeln anyway. Instead, we should use assert to clearly 
describe what happens:

// BAD: example of concatenation
string s1 = "Hello, ";
string s2 = "world!";
writeln(s1 ~ s2);  // writes Hello, world!


// GOOD: example of concatenation
string s1 = "Hello, ";
string s2 = "world!";
assert(s1 ~ s2 == "Hello, world!"); // no need for comment


Andrei