tree-sitter: parser generator tool and an incremental parsing library

[Vladimir Panteleev]

On Friday, 26 February 2021 at 20:06:46 UTC, Jacob Carlborg wrote:
> I wonder how the text editor/IDE integration is supposed to 
> work. To build the parser you need Node.js and a C compiler. Is 
> the author of the grammar supposed to pre-compile everything 
> and it's loaded as a dynamic library?

Yes.

> That would defiantly make it less flexible that the current 
> approach TextMate grammars. TextMate grammars are very easy to 
> modify on the fly and the editor will immediately be able to 
> use the new grammar. No building, no compilation and no loading 
> of dynamic libraries. But I do know how limiting TextMate 
> grammars are.

Yes, there is definitely an edit-compile-test cycle.

The D grammar I am working on currently takes about 15 seconds to 
compile.

> I guess in theory the editor could ship with Node.js and a C 
> compiler and compile on the fly to have the same flexibility as 
> TextMate grammars.

The great majority of users won't need to edit the grammar, so I 
think this is not an important goal to aim for. Parsing speed 
would probably be better appreciated than the ability to edit the 
grammar on the fly.

> BTW, I'm wondering if it needs to be Node.js or if any 
> JavaScript engine would work. At least macOS already ships with 
> a JavaScript engine.

I believe the tool chain is built on node/npm.

Grammar authors seem to keep the generated C files in git, so 
running the JavaScript part isn't needed to get the usable .so 
file.

For the future, I understand that the plan is to target 
WebAssembly instead. WebAssembly binaries are portable and it 
executes sufficiently quickly.