TS
RSS

Monday, 1 January 2000

What exactly is a `__pycache__`?

TODO

What exactly is a __pycache__?

While working on a Python project you may have noticed these folders that show up sometimes, named __pycache__.

To see how they show up, let's take a project I was working on recently: interpreted.

If you were to clone the project, you'd get this project structure:

.
├── src
│   └── interpreted
│       ├── __init__.py
│       ├── __main__.py
│       ├── cli.py
│       ├── interpreter.py
│       ├── nodes.py
│       ├── parser.py
│       ├── py.typed
│       └── tokenizer.py
├── tests
│   └── [...]
├── LICENSE
├── README.md
├── setup.cfg
├── setup.py
└── tox.ini
The `tests` folder has been truncated for brevity.

All the package's source code is present in the src/interpreted directory. The directory contains many python files: __init__.py, cli.py, parser.py and so on.

Now, if you were to start working on the project, by installing it and running tests:

# create a virtual environment before this
pip install .
pytest

pytest would run all the tests, and they will likely pass.

But, if you were to look at your directory structure again, it would look like this:

.
├── src
│   └── interpreted
│       ├── __init__.py
│       ├── __main__.py
│       ├── __pycache__
│       │   ├── __init__.cpython-311.pyc
│       │   ├── __main__.cpython-311.pyc
│       │   ├── cli.cpython-311.pyc
│       │   ├── interpreter.cpython-311.pyc
│       │   ├── nodes.cpython-311.pyc
│       │   ├── parser.cpython-311.pyc
│       │   └── tokenizer.cpython-311.pyc
│       ├── cli.py
│       ├── interpreter.py
│       ├── nodes.py
│       ├── parser.py
│       ├── py.typed
│       └── tokenizer.py
├── tests
│   └── [...]
├── LICENSE
├── README.md
├── setup.cfg
├── setup.py
└── tox.ini

Notice the new __pycache__ folder. Can you spot the correlation?

Indeed, for every .py folder inside src/interpreted, there is a .python-311.pyc file in the __pycache__ folder.

For some spoilers, yeah this is indeed a cache, created for each of these Python files. But what kind of cache, exactly?

pyc files: the what and the why

sidetrack: Python is compiled?

Creating a pyc file

Running a pyc file

Appendix: hash based cache checking?