Showing unittest in documentation (Was Re: std.unittests

[Jonathan M Davis]

On Monday 24 January 2011 15:27:08 foobar wrote:
> Steven Schveighoffer Wrote:
> > On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu
> > 
> > <SeeWebsiteForEmail at erdani.org> wrote:
> > > On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:
> > >> On 1/24/11 1:50 PM, Jens Mueller wrote:
> > >>> Jonathan M Davis wrote:
> > >>>> I think that it's been discussed a time or two, but nothing has been
> > >>>> done about
> > >>>> it. It wouldn't be entirely straightforward to do. Essentially,
> > >>>> either a
> > >>>> unittest block would have to be generated from the Examples section
> > >>>> in the
> > >>>> documentation, or you'd have to have some way to indicate that a
> > >>>> particular
> > >>>> unittest block got put into the documentation as an Examples
> > >>>> section. It's
> > >>>> certainly true that it would be ideal to have a way to avoid the
> > >>>> duplication,
> > >>>> but we don't have one at the moment, and it hasn't yet been a high
> > >>>> enough
> > >>>> priority to sort out how to do it and implement it.
> > >>> 
> > >>> I see. I understand that it does not have high priority. Just
> > >>> wondered whether ...
> > >>> 
> > >>> Jens
> > >> 
> > >> The change is much simpler than what Jonathan suggests. A change can
> > >> be made such that any unittest preceded by a documentation comment is
> > >> automatically considered an example.
> > >> 
> > >> /**
> > >> Example:
> > >> */
> > >> unittest
> > >> {
> > >> writeln("This is how it works.");
> > >> }
> > >> 
> > >> 
> > >> Andrei
> > > 
> > > BTW I consider this a very important topic. We have _plenty_ of
> > > examples that don't work and are not mechanically verifiable. The
> > > reasons range from minor typos to language changes to implementation
> > > limitations. Generally this is what they call "documentation rot".
> > > This is terrible PR for the language.
> > > 
> > > Changing ddoc to recognize documentation unittests would fix this
> > > matter once and forever.
> > > 
> > > Last but not least, the "----" separators for code samples are awful
> > > because no editor recognizes them for anything - they confuse the hell
> > > out of Emacs for one thing.
> > 
> > This only makes sense if:
> > 
> > 1. The unit test immediately follows the item being documented
> > 2. The unit test *only* tests that item.
> > 
> > The second one could be pretty annoying.  Consider cases where several
> > functions interact (I've seen this many times on Microsoft's
> > Documentation), and it makes sense to make one example that covers all of
> > them.  Having them 'testable' means creating several identical unit
> > tests.
> > 
> > One way to easily fix this is to allow an additional parameter to the
> > comment:
> > 
> > /**
> > Example(Foo.foo(int), Foo.bar(int)):
> > */
> > unittest
> > {
> > 
> >     auto foo = new Foo;
> >     foo.foo(5);
> >     foo.bar(6);
> >     assert(foo.toString() == "bazunga!");
> > 
> > }
> > 
> > The above means, copy the example to both Foo.foo(int) and Foo.bar(int)
> > 
> > An alternative that is more verbose, but probably more understandable:
> > 
> > /**
> > Example:
> > Covers Foo.foo(int)
> > Covers Foo.bar(int)
> > */
> > 
> > Of course, a lack of target just means it applies to the item just
> > documented.
> > 
> > One other thing, using writefln is considered bad form in unit tests (you
> > want *no* output if the unit test works).  But many examples might want
> > to demonstrate how e.g. an object interacts with writefln.  Any
> > suggestions? The assert line above is not very pretty for example...
> > 
> > -Steve
> 
> Unit-tests are defined on a module level, not a function level, hence I
> would expect to see the unit-tests that serve as examples to appear in an
> examples section on the page the documents the module itself. In the
> online docs I would expect the function names used in the example to be
> links to the individual function doc and for each function have a list of
> links to examples it participated in. This should be automatic and the
> user shouldn't need to provide the "list of functions documented in this
> example".
> 
> Just my two cents..

Examples almost always go with functions. The whole point is to avoid having to 
have the function examples both in the documentation and in a unittest block. It 
would be a huge mess to try and put all of the examples in the module 
documentation. I shudder to think what that would look like for std.algorithm - 
or even worse (_far_ worse), std.datetime. The simple syntax of

/++ Example +/
unittest
{
}


making that unittest block go in the documentation of the preceding function 
should work just fine. We already have /++ Ditto +/ which puts a function in with 
the preceding function's documentation. So, having /++ Example +/ on top of that 
isn't much of a stretch.

- Jonathan M Davis