Fixing dub search

aberba karabutaworld at gmail.com
Tue Dec 29 08:45:02 UTC 2020


On Monday, 28 December 2020 at 20:49:12 UTC, sarn wrote:
> On Monday, 28 December 2020 at 10:49:58 UTC, aberba wrote:
>> To fix this, a real search engine is needed. ElasticSearch is 
>> an overkill for what we need... basic accurate string search.
>>
>> Solution: use MeiliSearch. It's very lightweight and fast (1GB 
>> vps is more than enough). Very easy to use... just a REST API 
>> call. I already have a package for meilisearch on dub.
>
> ElasticSearch also has a simple REST API and would do this job 
> on whatever hardware we'd realistically use.  I'm not a huge ES 
> fan, personally, but do you have more reasons to dismiss it as 
> overkill and recommend MeiliSearch instead?
>
> The best place for discussion is here, though:
> https://github.com/dlang/dub-registry/issues/93
>
> But I have to say something again: please, please, please, I 
> beg, consider using an embedded search tool before adding an 
> external server (or, worse, an external SaaS) as a runtime 
> dependency to the dub registry.  There are only a few thousand 
> packages, and they don't update much.  Even grepping the whole 
> dataset every request would be fast enough (just not featureful 
> enough).

If you've looked at the very discussion you referenced, you'd 
realize they went around and still came back to using mongodb for 
search.

Not only is elasticsearch built in Java, hence bloatware, it's 
also designed to do more than just search... hence more bloatware 
and overkill for just basic search. You may compare the size of 
elastic with meilisearch which is just a small binary. 
MeiliSearch can run on very little ram...

The accuracy you'd get from a search engine just isn't possible 
with brute-force and hacks. Search is a complex problem involving 
stemming, plurals, synonyms, step words, ranking, etc. You'd want 
to use a real search engine.

And between elasticsearch and MeiliSearch, MeiliSearch is 
simpler, lightweight and easy to use.



More information about the Digitalmars-d mailing list