Constantly monitor a program/process using Python
cron-task, daemon, python, scheduling
Solution
There are many programs that can do this.
On Ubuntu there is upstart (installed by default)
Lots of people like http://supervisord.org/
monit as mentioned by @nathan
If you are looking for a python alternative there is a library that has just been released called circus which looks interesting.
And pretty much every linux distro probably has one of these built in.
The choice is really just down to which one you like better, but you would be far better off using one of these than writing it yourself.
Hope that helps
Problem
I am trying to constantly monitor a process which is basically a Python program. If the program stops, then I have to start the program again. I am using another Python program to do so. For example, say I have to constantly run a process called `run_constantly.py`. I initially run this program manually, which writes its process ID to the file "PID" (in the location out/PROCESSID/PID). Now I run another program which has the following code to monitor the program `run_constantly.py` from a Linux environment: ``` def Monitor_Periodic_Process(): TIMER_RUNIN = 1800 foo = imp.load_source("Run_Module","run_constantly.py") PROGRAM_TO_MONITOR = ['run_constantly.py','out/PROCESSID/PID'] while(1): # call the function checkPID to see if the program is running or not res = checkPID(PROGRAM_TO_MONITOR) # if res is 0 then program is not running so schedule it if (res == 0): date_time = datetime.now() scheduler.add_cron_job(foo.Run_Module, year=date_time.year, day=date_time.day, month=date_time.month, hour=date_time.hour, minute=date_time.minute+2) scheduler.start() scheduler.get_jobs() time.sleep(TIMER_NOT_RUNIN) continue else: #the process is running sleep and then monitor again time.sleep(TIMER_RUNIN) continue ``` I have not included the `checkPID()` function here. `checkPID()` basically checks if the process ID still exists (i.e. if the program is still running) and if it does not exist, it returns `0`. In the above program, I check if `res == 0`, and if so, then I use Python's scheduler to schedule the program. However, the major problem that I am currently facing is that the process ID of this program and the `run_constantly.py` program turns to be same once I schedule the `run_constantly.py` using the `scheduler.add_cron_job()` function. So if the program `run_constantly.py` crashes, the following program still thinks that the `run_constantly.py` is running (since both process IDs are same), and therefore continues to go into the else loop to sleep and monitor again. Can someone tell me how to solve this issue? Is there a simple way to constantly monitor a program and reschedule it when it has crashed?