Showing unittest in documentation (Was Re: std.unittests [updated] for review)

[Steven Schveighoffer]

On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail at erdani.org> wrote:

> On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:
>> On 1/24/11 1:50 PM, Jens Mueller wrote:
>>> Jonathan M Davis wrote:
>>>> I think that it's been discussed a time or two, but nothing has been
>>>> done about
>>>> it. It wouldn't be entirely straightforward to do. Essentially,  
>>>> either a
>>>> unittest block would have to be generated from the Examples section
>>>> in the
>>>> documentation, or you'd have to have some way to indicate that a
>>>> particular
>>>> unittest block got put into the documentation as an Examples section.
>>>> It's
>>>> certainly true that it would be ideal to have a way to avoid the
>>>> duplication,
>>>> but we don't have one at the moment, and it hasn't yet been a high
>>>> enough
>>>> priority to sort out how to do it and implement it.
>>>
>>> I see. I understand that it does not have high priority. Just wondered
>>> whether ...
>>>
>>> Jens
>>
>> The change is much simpler than what Jonathan suggests. A change can be
>> made such that any unittest preceded by a documentation comment is
>> automatically considered an example.
>>
>> /**
>> Example:
>> */
>> unittest
>> {
>> writeln("This is how it works.");
>> }
>>
>>
>> Andrei
>
> BTW I consider this a very important topic. We have _plenty_ of examples  
> that don't work and are not mechanically verifiable. The reasons range  
> from minor typos to language changes to implementation limitations.  
> Generally this is what they call "documentation rot". This is terrible  
> PR for the language.
>
> Changing ddoc to recognize documentation unittests would fix this matter  
> once and forever.
>
> Last but not least, the "----" separators for code samples are awful  
> because no editor recognizes them for anything - they confuse the hell  
> out of Emacs for one thing.

This only makes sense if:

1. The unit test immediately follows the item being documented
2. The unit test *only* tests that item.

The second one could be pretty annoying.  Consider cases where several  
functions interact (I've seen this many times on Microsoft's  
Documentation), and it makes sense to make one example that covers all of  
them.  Having them 'testable' means creating several identical unit tests.

One way to easily fix this is to allow an additional parameter to the  
comment:

/**
Example(Foo.foo(int), Foo.bar(int)):
*/
unittest
{
    auto foo = new Foo;
    foo.foo(5);
    foo.bar(6);
    assert(foo.toString() == "bazunga!");
}

The above means, copy the example to both Foo.foo(int) and Foo.bar(int)

An alternative that is more verbose, but probably more understandable:

/**
Example:
Covers Foo.foo(int)
Covers Foo.bar(int)
*/

Of course, a lack of target just means it applies to the item just  
documented.

One other thing, using writefln is considered bad form in unit tests (you  
want *no* output if the unit test works).  But many examples might want to  
demonstrate how e.g. an object interacts with writefln.  Any suggestions?   
The assert line above is not very pretty for example...

-Steve