Nginx and php-fpm: cannot get rid of 502 and 504 errors

Question

I have an ubuntu-server and a pretty high loaded website. Server is: - Dedicated to nginx, uses php-fpm (no apache), mysql is located on different machine - Has 8 GB of RAM - Gets about 2000 requests per second. Each php-fpm process consumes about 65MB of RAM, according to `top` command: Free memory: ``` admin@myserver:~$ free -m total used free shared buffers cached Mem: 7910 7156 753 0 284 2502 -/+ buffers/cache: 4369 3540 Swap: 8099 0 8099 ``` PROBLEM Lately, I'm experiencing big performance problems. Very big response times, very many `Gateway Timeouts` and in evenings, when load gets high, 90% of the users just see "Server not found" instead of the website (I cannot seem to reproduce this) LOGS My Nginx error log is full of the fallowing messages: ``` 2012/07/18 20:36:48 [error] 3451#0: *241904 upstream prematurely closed connection while reading response header from upstream, client: 178.49.30.245, server: example.net, request: request: "GET /readarticle/121430 HTTP/1.1", upstre…

Accepted Answer

A number of issues. Still worth to fix them with such a busy site. MySQL may be the root cause for now. But longer term you need to do more work.

Caching

I see one of your error msg showing a get request to the php upstream. This doesn't look good with such a high traffic site (2000 r/s as you mentioned). This page (/readarticle/121430) seems a perfectly cacheable page. For one, you can use nginx for caching such pages. Check out fastcgi cache

```
GET /readarticle/121430
```

php-fpm

```
pm.max_requests = 100
```

The value means that a process will be killed by php-fpm master after serving 100 requests. php-fpm uses that value to fight against 3rd party memory leaks. Your site is very busy, with 2000r/s. Your max child processes is 130, each can only serve at most 100 requests. That means after 13000/2000 = 6.5 seconds all of them are going to be recycled. This is way too much (20 processes killed every second). You should at least start with a value of 1000 and increase that number as long as you don't see memory leak. Someone uses 10,000 in production.

nginx.conf

Issue 1:

```
    if (!-e $request_filename) {
        rewrite ^(.*)$ /index.php?path=$1 last;
    }
```

should be replaced by more efficient try_files:

```
    try_files $uri /index.php?path=$uri;
```

You save an extra if location block and a regex rewrite rule match.

Issue 2: using unix socket will save you more time than using ip (around 10-20% from my experience). That's why php-fpm is using it as default.

Issue 3: You may be interested in setting up keepalive connections between nginx and php-fpm. An example is given here in nginx official site.

Nginx and php-fpm: cannot get rid of 502 and 504 errors

Solution

Problem