php/timeout/connection to server reset?

Question

I have a php script that needs to run for quite some time. What the script does: - connects to mysql - initiates anywhere from 100 to 100,000 cURL requests - each cURL request returns compact-decoded data of 1 to 2000 real estate listings - i use preg-match-all to get all the data and do one mysql insert per listing. each query never exceeds more than 1mb of data. So there are a lot of loops, mysql inserts, and curl requests going on. php safe mode is off and I am able to successfully ini_set the max-execution-time to something ridiculous to allow my script to run all the way through. Well, my problem is the script or apache or something is having a stroke in the middle of the script and the screen goes to the "connection to the server has been reset" screen. Any ideas?

Accepted Answer

Well, disregarding the fact that attempting 100,000 cURL requests is absolutely insane, you're probably hitting the memory limit.

Try setting the memory limit to something more reasonable:

```
ini_set('memory_limit', '256M');
```

And as a side tip, don't set the execution time to something ludicrous, chances are you'll eventually find a way to hit that with a script like this. ;]

Instead, just set it to `0`, it functionally equivalent to turning the execution limit off completely:

```
ini_set('max_execution_time', 0);
```

php/timeout/connection to server reset?

Solution

Problem