Recommended configuration for OpenSearch

[SergioCea]

Hi,

I am migrating from Solr to OpenSearch, I have the problem when configuring it for a large volume.

More than 10 million nodes, and in Solr it indicates that it has an index size of 120Gb, my doubt above all comes with the issue of the number of nodes to create in OpenSearch. Since with some small test I have seen that the searches take a while.

Thank you for the help

[LeoMattioli]

Sizing recommendations are always tricky, if possible, as they involve lots of variables: it's not much about number of docs but the size of the repo, if you need FTS or only Metadata search,...

Let's say that you  should aim at shards not being bigger than 32GB each, and have at least 1 replica for prod envs, with 3 master node. 

You can also check this post: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/sizing-domains.html 

Hope this helps.

[SergioCea]

Hello, why is the replica recommended? On the other hand, I index metadata and content, I understand that having 120gb is 5-6 shards for having margin. On the other hand, having 3 nodes, there are 3 instances. Before I had a Solr EC2 of 16 CPUs and 62GB so now those 3 instances should add that but what is better, 3 of r6g.large.search or 3 of r6g.xlarge.search?

[LeoMattioli]

Hi, have a look at this article that explain better than me about replicas and shards https://www.baeldung.com/java-shards-replicas-elasticsearch

About the instance type: usually "the more, the merrier", but you should look at the performances you want to achieve, I mean "the average response time when I do a search on metadata/full text search should be xxx seconds".

Hope this helps.