Freezing Package Requirements21 Jan 2016
I have a minor issue with freezing requirements, and so I put together a very complex solution. One that is documented here. Not 100% sure why this week is all about packaging, but there you go.
First up, what is a requirement file? Basically they are a list of items that can be installed with
pip using the following command:
$ pip install -r requirements.txt
The file therefore mostly serves as a list of arguments to the
pip install command. The requirements file itself has a very specific format and can be created by hand, but generally the
pip freeze command is used to dump out the requirements as follows:
$ pip freeze > requirements.txt
This produces an alphabetically sorted list of requirements and dumps them to your requirements text file. It is particularly useful when you are working in a virtual environment as you can track your project specific dependencies. Moreover, your
setup.py file can read the
requirements.txt and install dependencies via
INSTALL_REQUIRES. However, consider the following requirements.txt file from Confire:
## Confire requirements PyYAML==3.11 ## Testing requirements #coverage==3.7.1 #nose==1.3.3 #coveralls==0.5 ## Added by virtualenv #wsgiref==0.1.2
Here you can see that we have only one true dependency for Confire, PyYAML. The issue, however, is that we also have the development testing dependencies (nose, coverage, etc.) and then some weird other dependencies from virtualenv or from pip. We’ve also added comments and whitespace to make this more readable. We have to comment out the testing requirements, because those shouldn’t be installed when you
pip install confire but they should be listed so contributors know what to expect. Luckily,
pip freeze has us covered in terms of ordering, comments, and whitespace:
$ pip freeze -r requirements.txt > requirements-new.txt $ mv requirements-new.txt requirements.txt
However because the commented packages are skipped before review, you end up with the following:
## Confire requirements PyYAML==3.11 ## Testing requirements #coverage==3.7.1 #nose==1.3.3 #coveralls==0.5 ## Added by virtualenv #wsgiref==0.1.2 ## The following requirements were added by pip --freeze: coveralls==0.5 coverage==3.7.1 nose==1.3.3 wsgiref==0.1.2
Like I said, a minor beef. If you’ve added or upgraded a package, then you have to manually deal with all the commented dependencies. Therefore I created a script to help me with this issue as follows.
Basically I’ve stuck this file into
~/bin/requires and now I can simply do the following to get my requirements:
$ requires -o reqs.txt $ mv reqs.txt requirements.txt
The script automatically detects the local
requirements.txt file. This script is far from perfect, and the following things I would like to do:
- Overwrite the actual freeze method shown on GitHub to tighten things up.
- Deal with new line parsing, inline comments, and versioning a bit better.
- Output helpful hints to
But for me, this is solving a problem, which is great!