Monthly ArchiveFebruary 2018

Setup a “PROD” Elasticsearch instance on AWS EC2 server

Hello everyone!

I’m back ! 🙂

 

So, I’m working on setting up an Elasticsearch cluster (version 6.2.1) and I wanted to share some advices with you !

 

First of all, you need to have AWS account !! not difficult, you only need to connect here : https://aws.amazon.com/fr/

 

After that, you need to create a new EC2 instance (in my case):

• rhel 7.4 image
• 2 vCPU
• 4 GB RAM
• 20 GB storage

Now,  you need to download :

• Elasticsearch : https://www.elastic.co/downloads/elasticsearch (.tar.gz)
• JDK 8 : http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html (for linux 64x)

 

You can either download it directly from your EC2 instance (via wget command) or download it locally on your desktop and use wincp to copy those files to your EC2 instance.

 

Now, let’s untar elasticsearch and jdk:

• export JAVA_HOME=/home/ec2-user/jdk1.8.0_161 (“vi .bashrc” and “source .bashrc”
• here is the configuration of my Elasticsearch instance (config/elasticsearch.yml):
  • cluster.name: application-mhichri
  • node.name: node-1
  • path.data: /home/ec2-user/es-data
  • path.logs: /home/ec2-user/es-logs
  • network.host: 0.0.0.0
  • http.port: 9200

Now, we will try to start our elasticsearch instance:

• ./bin/elasticsearch

but! we are getting these 2 errors :

• [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
• [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Actually, since Elasticsearch 5, you cannot bind your elasticsearch instance to 0.0.0.0 only if you set these 2 params:

• vi (as root) /etc/security/limits.conf ==> add : “ec2-user – nofile 65536” (resolves [1])
• sudo sysctl -w vm.max_map_count=262144 (resolves [2])

It’s considered as a “PROD” instance if you bind your elasticsearch instance to 0.0.0.0. So, before it starts, Elasticsearch will check those 2 system configurations !

 

Now, we’re good !  we can start our Elasticsearch instance “./bin/elasticsearch -d” (on background)!

Check it here : http://ec2-184-72-102-44.compute-1.amazonaws.com:9200/

 

PS : don’t forget to check in your security group policies, that you allowed http communication on port 9200 !