Amazon EC2 ELB alarm - which instance is unhealthy?

Question

We have hosted some apps on Amazon EC2 and are using an Elastic Load Balancer (ELB) to manage several instances of one app. Also, we have set up ELB alarms to get notified about Unhealthy Hosts, i.e. when an instance has gone down. So far, I could not figure out where to check which instance exactly has gone down when the alarm goes off, except for the ELB status page in the AWS console. However, if the instance comes back to In Service state again, this won't help me either. The e-mail notification sent out by the ELB does not contain this information; and I couldn't find it anywhere in the alarms history in the console either. Is there a way to tell which instance an ELB alarm has been triggered for, even if the instance has come back into OK state in the meantime? Cheers, Alex

Accepted Answer

Sadly Amazon does not provide a health check log, so its impossible to find out which instance failed the health check afterwards, assuming that the server is no longer unhealthy. You can only use Per-Az metrics to know in which AZ is the instance.

But, you could know which instance is down if you query AWS api during the issue. So, I have thought of a possible workaround:

- Set up a new SNS topic, and add an HTTP action to a custom URL that triggers a job that enumerates the instances and send you that info by mail.

- Then setup a CloudWatch alarm for UnHealthyHostCount > 0 and setup the action to the SNS topic.

The difficult part is that your URL should handle the SNS subscription & confirmation described here.

The command to know which instance is currently OutOfService is:

```
elb-describe-instance-health *LoadBalancerName* --region *YourRegion*
```

Amazon EC2 ELB alarm - which instance is unhealthy?

Solution

Problem