python does not release filehandles to logfile

Question

I have an application which has to run a number of simulation runs. I want to setup a logging mechanisme where all logrecords are logged in a general.log, and all logs for a simulation run go to run00001.log, .... For this I have defined a class Run. in the `__init__()` a new filehandle is added for the runlog. The problem is that the logfiles for the runs never get released, so after a number of runs the available handles are exhausted and the run crashes. I've set up some routines to test this as follows main routine ``` import Model try: myrun = Model.Run('20130315150340_run_49295') ha = raw_input('enter') myrun.log.info("some info") except: traceback.print_exc(file=sys.stdout) ha = raw_input('enter3') ``` The class Run is defined in module Model as follows ``` import logging class Run(object): """ Implements the functionality of a single run. """ def __init__(self, runid): self.logdir="." self.runid = runid self.logFile = os.path.join(self.logdir , self.runid + '.log') self.log =…

Accepted Answer

You need to call `.close()` on the filehandler.

When your `Run` class completes, call:

```
handlers = self.log.handlers[:]
for handler in handlers:
    self.log.removeHandler(handler)
    handler.close()
```

A file handler will automatically re-open the configured filename every time a new log message arrives, so calling `handler.close()` may sometimes appear futile. Removing the handler from the logger stops future log records from being sent to it; in the above code we do this first, to avoid an untimely log message from another thread reopening the handler.

Another answer here suggest you use `logging.shutdown()`. However, all that `logging.shutdown()` does is call `handler.flush()` and `handler.close()`, and I'd not recommend using it. It leaves the logging module in a state where you can't use `logging.shutdown()` again, not reliably.

python does not release filehandles to logfile

Solution

Problem