How do I check which files are open or closed in Python

Question

Hi all,

I am stuck in a situation where I am being presented with an error that I have too many files open. This program of mine has to run for a long time and hence the number of files is more.

My question is that if there is any particular way that I am missing which will directly help me keep track of what files are open and which are closed.

This will basically help me to print it out in a list and see where the problem actually lies.

All help appreciated!

Answer 1

Hi, good question. I have a solution for you. 

So what I have done here is that I have wrapped a built-in file object at the starting point of the program itself. 

What does this do?

Well, it basically told me on the whole that the loggers were not being closed which ended up resulting in the error.

Check out the program below, it is very simple:

import io
import sys
import builtins
import traceback
from functools import wraps


def opener(old_open):
    @wraps(old_open)
    def tracking_open(*args, **kw):
        file = old_open(*args, **kw)

        old_close = file.close
        @wraps(old_close)
        def close():
            old_close()
            open_files.remove(file)
        file.close = close
        file.stack = traceback.extract_stack()

        open_files.add(file)
        return file
    return tracking_open


def print_open_files():
    print(f'### {len(open_files)} OPEN FILES: [{", ".join(f.name for f in open_files)}]', file=sys.stderr)
    for file in open_files:
        print(f'Open file {file.name}:\n{"".join(traceback.format_list(file.stack))}', file=sys.stderr)


open_files = set()
io.open = opener(io.open)
builtins.open = opener(builtins.open)

This should work like a charm, hope this helped!