Streaming parsers in D a novel design

[H. S. Teoh]

On Tue, Apr 18, 2023 at 08:13:05AM +0000, Dmitry Olshansky via Digitalmars-d wrote:
> A streaming parser is doing 2 things:
> - waiting for input
> - processing the current input and spitting out Events
> 
> This maps to D beautifully:
> 
> 1. Waiting for input means it's an OutputRange with explicit put
> method!
> 
> 2. Processing events means it's an InputRange of events. It may even
> be ForwardRange, mine first of this kind is Forward range.
[...]

In practice, the input would be an input range consumed by the parser
which returns an input range of tokens.  IOW the parser is a filter that
transforms an input range of characters into a range of tokens.

You'd only need an output range interface if you needed to explicitly
feed characters to the parser, such as if you were using an async I/O
API and you just received a block of fresh input, and now you need to
stuff the input into the parser.  But even then, I wouldn't do it this
way, because on the other end, the input range wouldn't be able to
return a result if, say, the next block of input hasn't arrived yet. So
.empty would have to block, which is an ugly design for something with
an input range API.

At the end of the day, I think treating a parser as a transformation
(range of char -> range of tokens) is a better abstraction than trying
to shoehorn it into something involved output ranges.


T

-- 
Nearly all men can stand adversity, but if you want to test a man's character, give him power. -- Abraham Lincoln