How to scale with massive update queries in Elasticsearch?

Introduction

What this talk is all about?

We recently moved from MySQL to Elasticsearch where we got a direct 10x - 15x boost in our performance.

We came up with unique use cases of heavy updates in Elasticsearch. That been challenging but yes currently Our Elaticsearch handling 200 million requests per day very efficiently. Our WRITE consist of the partial update, update with script conditions and of course simple indexing.

Our READ request is 1 million/day which contains Scroll, simple search & Aggregations Query. We achieved to display our email logs in next to real-time. We also worked on Disk optimization by believing in the principle of “Know your query”. Currently having 6TB + of the cluster with 80 GB ingestion per day.

Slides

Talk Video

Feel free to comment below, If you have any doubts or suggestion about this talk.