Elasticsearch : Connection refused while connecting to upstream

elasticsearch, kibana, nginx

Solution

Seems like `upstream` and a different `keepalive` is necessary for the ES backend to work properly, I finally had it working using the following configuration :

upstream elasticsearch {
    server 127.0.0.1:9200;
    keepalive 64;
}

server {

  listen 8080;
  server_name myserver.com;
  error_log   /var/log/nginx/elasticsearch.proxy.error.log;
  access_log  off;

  location / {

    # Deny Nodes Shutdown API
    if ($request_filename ~ "_shutdown") {
      return 403;
      break;
    }

    # Pass requests to ElasticSearch
    proxy_pass http://elasticsearch;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Connection "";

    proxy_set_header  X-Real-IP  $remote_addr;
    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header  Host $http_host;

    # For CORS Ajax
    proxy_pass_header Access-Control-Allow-Origin;
    proxy_pass_header Access-Control-Allow-Methods;
    proxy_hide_header Access-Control-Allow-Headers;
    add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type';
    add_header Access-Control-Allow-Credentials true;

  }

}

Problem

I've set up an Elasticsearch server with Kibana to gather some logs. Elasticsearch is behind a reverse proxy by Nginx, here is the conf : ``` server { listen 8080; server_name myserver.com; error_log /var/log/nginx/elasticsearch.proxy.error.log; access_log off; location / { # Deny Nodes Shutdown API if ($request_filename ~ "_shutdown") { return 403; break; } # Pass requests to ElasticSearch proxy_pass http://localhost:9200; proxy_redirect off; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; # For CORS Ajax proxy_pass_header Access-Control-Allow-Origin; proxy_pass_header Access-Control-Allow-Methods; proxy_hide_header Access-Control-Allow-Headers; add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type'; add_header Access-Control-Allow-Credentials true; } } ``` Everything works well, I can `curl -XGET "myserver.com:8080"` to check, and my logs come in. But every minute or so, in the nginx error logs, I get that : ``` 2014/05/28 12:55:45 [error] 27007#0: *396 connect() failed (111: Connection refused) while connecting to upstream, client: [REDACTED_IP], server: myserver.com, request: "POST /_bulk?replication=sync HTTP/1.1", upstream: "http://[::1]:9200/_bulk?replication=sync", host: "myserver.com" ``` I can't figure out what it is, is there any problem in the conf that would prevent some `_bulk` requests to come through ?

Original source