Showing unittest in documentation (Was Re: std.unittests [updated] for review)

Steven Schveighoffer schveiguy at yahoo.com
Mon Jan 24 13:36:21 PST 2011


On Mon, 24 Jan 2011 16:03:24 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail at erdani.org> wrote:

> On 1/24/11 2:37 PM, Steven Schveighoffer wrote:
>> On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu
>> <SeeWebsiteForEmail at erdani.org> wrote:
>>
>>> On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:
>>>> On 1/24/11 1:50 PM, Jens Mueller wrote:
>>>>> Jonathan M Davis wrote:
>>>>>> I think that it's been discussed a time or two, but nothing has been
>>>>>> done about
>>>>>> it. It wouldn't be entirely straightforward to do. Essentially,
>>>>>> either a
>>>>>> unittest block would have to be generated from the Examples section
>>>>>> in the
>>>>>> documentation, or you'd have to have some way to indicate that a
>>>>>> particular
>>>>>> unittest block got put into the documentation as an Examples  
>>>>>> section.
>>>>>> It's
>>>>>> certainly true that it would be ideal to have a way to avoid the
>>>>>> duplication,
>>>>>> but we don't have one at the moment, and it hasn't yet been a high
>>>>>> enough
>>>>>> priority to sort out how to do it and implement it.
>>>>>
>>>>> I see. I understand that it does not have high priority. Just  
>>>>> wondered
>>>>> whether ...
>>>>>
>>>>> Jens
>>>>
>>>> The change is much simpler than what Jonathan suggests. A change can  
>>>> be
>>>> made such that any unittest preceded by a documentation comment is
>>>> automatically considered an example.
>>>>
>>>> /**
>>>> Example:
>>>> */
>>>> unittest
>>>> {
>>>> writeln("This is how it works.");
>>>> }
>>>>
>>>>
>>>> Andrei
>>>
>>> BTW I consider this a very important topic. We have _plenty_ of
>>> examples that don't work and are not mechanically verifiable. The
>>> reasons range from minor typos to language changes to implementation
>>> limitations. Generally this is what they call "documentation rot".
>>> This is terrible PR for the language.
>>>
>>> Changing ddoc to recognize documentation unittests would fix this
>>> matter once and forever.
>>>
>>> Last but not least, the "----" separators for code samples are awful
>>> because no editor recognizes them for anything - they confuse the hell
>>> out of Emacs for one thing.
>>
>> This only makes sense if:
>>
>> 1. The unit test immediately follows the item being documented
>> 2. The unit test *only* tests that item.
>
> That's the what current examples do for virtually all of Phobos.

#1 applies to all current examples, but I was also thinking of cases where  
current unit tests may also be turned into examples.

The second in some cases is not true.  Let's find an example in  
std.algorithm:

http://www.digitalmars.com/d/2.0/phobos/std_algorithm.html#partition

This example is very long and contains lots of std.algorithm functions  
besides partition.  With some slight tweaking, you could cleanly cover  
multiple functions in one "Example".  Such examples can be more  
instructive because they are more like real code than two or three assert  
tests.

>> The second one could be pretty annoying. Consider cases where several
>> functions interact (I've seen this many times on Microsoft's
>> Documentation), and it makes sense to make one example that covers all
>> of them. Having them 'testable' means creating several identical unit
>> tests.
>>
>> One way to easily fix this is to allow an additional parameter to the
>> comment:
>>
>> /**
>> Example(Foo.foo(int), Foo.bar(int)):
>> */
>> unittest
>> {
>> auto foo = new Foo;
>> foo.foo(5);
>> foo.bar(6);
>> assert(foo.toString() == "bazunga!");
>> }
>>
>> The above means, copy the example to both Foo.foo(int) and Foo.bar(int)
>
> Why would I force the reader to read the same example twice? And why  
> would I run the same unittest twice?

Because you have a very illustrative example, and it applies to more than  
one function.  Why would you run it twice?  Because your proposed system  
*forces* you to run it twice.

>
>> An alternative that is more verbose, but probably more understandable:
>>
>> /**
>> Example:
>> Covers Foo.foo(int)
>> Covers Foo.bar(int)
>> */
>>
>> Of course, a lack of target just means it applies to the item just
>> documented.
>
> I find documented unittests attractive mainly because they're _simple_.  
> As soon as we start to add that kind of stuff... exponential decay.

It's only not simple if you want it to be.  The /** Example: */ simple  
method is also covered.  Let's also not forget that the end result is  
generated documentation, not the comments.  All this 'non-simplicity' is  
going to be hidden there.

The point is, not everyone writes *unique* examples for each of their  
functions.  I think a detailed example that demonstrates multiple  
functions can be more informative than one that shows a simple usage.   
This does not apply to all functions/types, and those functions would not  
require "targeted" unit tests, just the simple ddoc designation.

>
>> One other thing, using writefln is considered bad form in unit tests
>> (you want *no* output if the unit test works). But many examples might
>> want to demonstrate how e.g. an object interacts with writefln. Any
>> suggestions? The assert line above is not very pretty for example...
>
> Yah, that is an issue. For examples that do non-unittesty stuff (e.g.  
> writeln, use sockets etc.) we can still use the old-style documentation.

That sounds reasonable, although I still think we need to be able to  
compile these to prevent doc rot.

> By the way, for all examples that don't explicitly describe writeln, we  
> shouldn't use writeln anyway. Instead, we should use assert to clearly  
> describe what happens:
>
> // BAD: example of concatenation
> string s1 = "Hello, ";
> string s2 = "world!";
> writeln(s1 ~ s2);  // writes Hello, world!
>
>
> // GOOD: example of concatenation
> string s1 = "Hello, ";
> string s2 = "world!";
> assert(s1 ~ s2 == "Hello, world!"); // no need for comment

I dunno, I sort of like the doc style from things like php, where it shows  
you the output in a separate box.

If DIP9 is accepted (writeTo), then showing examples of how the format  
specifiers work would certainly look less confusing via writefln.

It's not too important, but just something to think about.

-Steve


More information about the Digitalmars-d mailing list