More information about the Underscore mailing list

[_] Database choice for a social media platform

Erik Wallace erik.wallace at lewdewe.com
Thu May 19 10:41:15 BST 2016

Hi Oliver,

Thanks for your input. As you say, it's easy to store mountains of data but accessing it and finding what you want quickly is the main problem. ElasticSearch looks interesting. When you split storage and search in this manner does it require two copies of the data, one in each database system, or is ES accessing the data stored in MySQL/MariaDB?


Kind Regards,
Erik Wallace
________________________________________
From: Underscore <underscore-bounces at under-score.org.uk> on behalf of Oliver Humpage <oliver at watershed.co.uk>
Sent: 19 May 2016 10:29:52
To: underscore at under-score.org.uk
Subject: Re: [_] Database choice for a social media platform

> On 19 May 2016, at 10:09, Rick Hurst <rick.hurst at gmail.com> wrote:
>
> This is where Elasticsearch et al come in.

Yes - search indexing and data storage are very different problems. Even we, on our tiny watershed.co.uk site, separate out search and data. So when you load a page, it comes from standard MySQL. But when you search, it uses elasticsearch - partly because it’s lightning fast, but mostly because if you set up your initial document mappings and your queries right you can get really good results (things like ngrams for titles and names are very handy).

Also clustering just seems to work with ES - we’ve (accidentally) thrown all kinds of real-world hazards at our heavily-used logging cluster and never once had data loss or inconsistency.

Oliver.

--
underscore_ list info/archive -> http://www.under-score.org.uk/mailman/listinfo/underscore