Sony Arouje

The system I was working might need more than one Elasticsearch node configured as Master or Load balancer. I wanted to come up with a logic to connect to a different node in case of node failure. That means I have to write logic to check the status of a node before sending any index or search request. If the status failed, take the next node from the configured list and check the status. This is a viable approach but more concerned about the performance impact of my logic.

I started searching for a system that can open up a proxy IP and group the elasticsearch instance’s under that ip. This way the client will be aware of this proxy ip and not the individual ip and port details of Elasticsearch nodes. Also any node failures will be handled by the proxy system. After some exploration I decided to try nginx, it has built in load balancing with very simple configuration. I downloaded the mainline version of nginx for Windows and installed it.

To modify the nginx configuration go to conf\nginx.conf and open the configuration file in any text editor and add your configuration. See my configuration below.

worker_processes  1;
events {
worker_connections  1024;
}

http {

include       mime.types;
default_type  application/octet-stream;

upstream elasticcluster{
server 192.168.11.105:9200 weight=1 fail_timeout=1;
server 192.168.11.105:9201 weight=1 fail_timeout=1;
}

sendfile        on;
keepalive_timeout  65;
server {
listen       8080;
server_name  127.0.0.1;

location / {
proxy_pass  http://elasticcluster;
proxy_connect_timeout 1;
}

error_page   500 502 503 504  /50x.html;
location = /50x.html {
root   html;
}

}
}

In the upstream I group all my Elasticsearch nodes under elasticcluster. In the server’s proxy_pass I specify the upstream name, eg. http://elasticcluster. That’s it, now I can access my elastic search node via 127.0.0.1:8080 and nginx will route the traffic to any one of my Elasticsearch node in round robin manner. As you can see nginx ip and port is configured via listen and server_name settings.

Now all my search or index request goes to nginx ip and nginx will route the traffic to any of the Elasticsearch node running in my dev machine. I test failover by stopping one of my Elasticsearch node and nginx routed the traffic to the other live nodes.

Issue with localhost

Initially I configured nginx and Elasticsearch to use different ports of localhost. This causes some huge delay in nginx to route traffic to Elasticsearch nodes. I found this post from nginx forum and realized that the issue is with localhost. So I configured all my Elasticsearch instance to use the ip address instead of localhost. Also configured nginx.conf and specify 127.0.0.1 as server_name, you can also give the IP address instead of 127.0.0.1.

Leave your comments below if you find it helpful or have any questions. Thanks for reading.

Happy coding…