BNF grammar for D?

[Bruno Medeiros via Digitalmars-d]

On 21/12/2014 00:34, Kingsley wrote:
> On Friday, 19 December 2014 at 02:53:02 UTC, Rikki Cattermole wrote:
>> On 19/12/2014 10:19 a.m., Kingsley wrote:
>>> On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote:
>>>>
>>>>> Hi Bruno,
>>>>>
>>>>> Thanks very much. I do have a couple of questions about DDT in
>>>>> relation to my plugin.
>>>>>
>>>>> Firstly - I'm not too familiar with parsing/lexing but at the moment
>>>>> the Psi Structure I have implemented that comes from the DDT
>>>>> parser/lexer is not in any kind of hierarchy. All the PsiElements are
>>>>> available but all at the same level. Is this how the DDT parser
>>>>> works? Or is it down to my implementation of the Parser/Lexer that
>>>>> wraps it to create some hierarchy.
>>>>>
>>>>> For intellij it's going to be vastly easier to have a hierarchy with
>>>>> nested elements in order to get hold of a structure representing a
>>>>> class or a function for example - in order to do things like get the
>>>>> start and end lines of a class definition in order to apply code
>>>>> folding and to use for searching for classes and stuff.
>>>>>
>>>>> Secondly - how active it the development of DDT - does it keep up
>>>>> with the D2 releases.
>>>>>
>>>>> --Kingsley
>>>>
>>>> After doing a bit more research it looks like I have to create the psi
>>>> hierarchy myself - my current psi structure is flat because I'm just
>>>> converting the DeeTokens into PsiElements directly. I've still got
>>>> some experimentation to do. On the plus side I implemented commenting,
>>>> code folding but everything else needs a psi hierarchy
>>>
>>> I've done some more investigation and I do need to build the parser
>>> myself in order to create the various constructs. I've made a start but
>>> I haven't gotten very far yet because I don't fully understand the
>>> correct way to proceed.
>>>
>>> I also had a look at using the DeeParser - because it already does most
>>> of what I want. However the intellij plugin wants a PsiParser which
>>> returns an intellij ASTNode in the primary parse method. I can't see an
>>> easy way to hook this up with DeeParser because the ParsedResult
>>> although had a node method on it - gives back the wrong type of ASTNode.
>>>
>>> Any pointers on how I might get the DeeParser to interface to an
>>> intellij ASTNode would be appreciated.
>>
>> Read my codebase again, it'll answer a lot of questions. Your parser
>> is different, but what it produces shouldn't be. and yes it supports
>> hierarchies.
>
> Hi
>
> So finally after a lot of wrestling with the internals of intellij I
> finally managed to get a working parser implementation that produces a
> psi hierarchy based on the DeeParser from the ddt code.
>
> The main issue was that Intellij only wants you to create a parser using
> their toolset - which is either with a BNF grammar that you can then
> generate the parser - or with a hand written parser. Since I'm already
> using the DDT lexer and there is a perfectly good DDT parser as well - I
> just wanted to re-use the DDT parser.
>
> However Intellij does not provide any way to create a custom AST/PSI
> structure or use an external parser. So I basically had to wrap the
> DeeParse inside the Intellij parser and sync them up programmatically.
> It's not the most efficient way in the world but it at least works.
>
> In the long term I will write a BNF grammar for Intellij (using their
> toolkit) but I can see that will take me several months so this is a
> quick way to get the plugin up and running with all the power of
> intellij extras without spending several months stuck learning all about
> the complexities of grammar parsing and lexing.
>
> Thanks very much for you help. Once I get a bit more of the cool stuff
> done I will release the plugin.

Again, I'm not familiar with Intellij internals or the PSI structure, so 
I don't know how the DDT parser can be adapted to that.

However, I do suspect there should be a way to create a PSI structure 
from the DDT ASTNode structure.

PS: Sorry for the long delay in replying, I don't often check the 
digitalmars.D newsgroup/forum, and can sometimes be behind on the posts 
there. If you want to grab my attention, better to post on 
digitalmars.D.ide as I keep a closer eye on that newsgroup.

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros