DCT: D compiler as a collection of libraries

[Marco Leise]

Am Sun, 20 May 2012 20:37:07 +0200
schrieb "Roman D. Boiko" <rb at d-coding.com>:

> >> > Since assembly code is usually small I just preallocate an 
> >> > array of sourceCode.length tokens and realloc it to the 
> >> > correct size when I'm done parsing. Nothing pretty, but 
> >> > simple and I am sure it won't get any faster ;).
> >> I'm sure it will :) (I'm going to elaborate on this some time 
> >> later).
> >
> > I'm curious.
> Maybe I'm don't understand your use case, but the idea is that if
> you parse as you type it should be possible to avoid parsing and
> allocating memory for those lines which have not changed. But
> that is not compatible with storing tokens in the array, since it
> would cause reallocating memory each time, so the other data
> structure should be used (e.g., a linked list, or, if efficient
> lookup is needed, a red-black tree). Only benchmarks can show
> whether (and how much) my approach would be faster for specific
> situation (input patterns like average size of data, complexity
> of parsing algorithms, requirements, etc.).

I just only thought about the single-use situation, not the situation when editing files. Now the idea has evolved a bit and you probably thought about storing the scope hierarchy in a tree, too. It is still difficult to write a fast parser when someone can add/remove a '{' somewhere at the top of datetime.d, which changes the interpretation of the rest of the file. Mono-D has some hickups worse than those (several seconds even) - maybe a bug. I'll keep my fingers crossed.

> If you want to pass a slice of input string to a function, you
> cannot append \0 to it without copying data. If you don't append
> some pre-defined character, you must check for length *and* all
> supported terminating characters. On the contrary, your design
> might not require passing slices, and if language syntax allows
> deterministic parsing (when you know what to expect next), there
> is no need for checking EoF.

Now I get it!

-- 
Marco