Proposal: Database Engine for D

[Piotrek via Digitalmars-d]

On Saturday, 2 January 2016 at 20:47:37 UTC, Chris Wright wrote:
> So you want to create the following query:
>
>   people.filter!(x => x.surname == "Slughorn");
>
> And you've got ten million people in the collection, and you 
> want this query to finish soonish. So you need to use an index. 
> But a full index scan isn't so great; you want to do an index 
> lookup if possible.


Correct.

> That's simple enough; we generate proxy types to record what 
> properties you're using and what operations you're performing. 
> PersonProxy records that you're accessing a field 'surname', 
> gives a StringFieldProxy, and that records that you're checking 
> for equality with the string "Slughorn". The lambda returns 
> true when opEquals returns true.
>
> But people write queries that are more complex than that, like:
>
>   people.filter!(x => x.surname == "Slughorn" || x.age <= 17);
>
> First time you run this, x.surname.opEquals("Slughorn") returns 
> true and the expression as a whole returns true. You missed the 
> second part of the expression. That's bad.
> So we need to evaluate this lambda twice per parameter.

Hmm. Probably we don't think about the "proxy" term in the same 
way.
When I mentioned proxy I thought about a skeleton object to 
access parts of the original heavy object. So in fact, filter 
works on proxy and one evaluation is performed no matter how 
complex the condition is. The proxy is responsible to retrieve 
needed data. Additionally (I haven't mentioned it yet) we can 
provide mechanism for data integrity when joining separate 
objects.

> You might be able to support queries as large as ten 
> comparisons in a reasonable timeframe. But for all but the most 
> trivial queries, it'll be faster to use SQL.

SQL is no magic. You can write a code to beat any SQL planner. 
Also I think you had "joins" in mind which I guess are the 
biggest problem for performance.

But "What if I tell there is no SQL" !!!! Not even under the hood.

Piotrek