From virtualenv, pip freeze > requirements.txt give TONES of garbage! How to trim it out?

deployment, django, heroku, pip, virtualenv

Solution

That is one thing that has bugged me too quite a bit. This happens when you create a virtualenv without the `--no-site-packages` flag.

There are a couple of things you can do:

- Create virtualenv with the `--no-site-packages` flag.

- When installing apps, dont run `pip install <name>` directly, instead, add the library to your `requirements.txt` first, and then install the requirements. This is slower but makes sure your requirements are updated.

- Manually delete libraries you dont need. A rule of thumb i follow for this is to add whatever is there in my `INSTALLED_APPS`, and database adapters. Most other required libraries will get installed automatically because of dependencies. I know its silly, but this is what I usually end up doing.

-- Edit --

I've since written a couple of scripts to help manage this. The first runs pip freeze and adds the found library to a provided requirements file, the other, runs pip install, and then adds it to the requirements file.

function pipa() {
    # Adds package to requirements file.
    # Usage: pipa <package> <path to requirements file>
    package_name=$1
    requirements_file=$2
    if [[ -z $requirements_file ]]
    then
        requirements_file='./requirements.txt'
    fi
    package_string=`pip freeze | grep -i $package_name`
    current_requirements=`cat $requirements_file`
    echo "$current_requirements\n$package_string" | LANG=C sort | uniq > $requirements_file
}

function pipia() {
    # Installs package and adds to requirements file.
    # Usage: pipia <package> <path to requirements file>
    package_name=$1
    requirements_file=$2
    if [[ -z $requirements_file ]]
    then
        requirements_file='./requirements.txt'
    fi
    pip install $package_name
    pipa $package_name $requirements_file
}

Problem

I'm following this tutorial: http://devcenter.heroku.com/articles/django At some point I'm suposed to do: ``` pip freeze > requirements.txt ``` (Ofc. from virtualenv created instance of python) And I get this: ``` (venv)przemoli@ubuntu:~/Programowanie/hellodjango$ cat requirements.txt BeautifulSoup==3.2.0 Brlapi==0.5.5 CherryPy==3.1.2 ClientForm==0.2.10 Django==1.3 GnuPGInterface==0.3.2 PAM==0.4.2 PIL==1.1.7 Routes==1.12.3 Twisted-Core==11.0.0 Twisted-Names==11.0.0 Twisted-Web==11.0.0 WebOb==1.0.8 adium-theme-ubuntu==0.3.1 apt-xapian-index==0.44 apturl==0.5.1ubuntu1 chardet==2.0.1 command-not-found==0.2.44 configglue==1.0 cssutils==0.9.8a1 defer==1.0.2 distribute==0.6.19 django-tagging==0.3.1 dnspython==1.9.4 duplicity==0.6.15 gnome-app-install==0.4.7-nmu1ubuntu2 httplib2==0.7.2 jockey==0.9.4 keyring==0.6.2 launchpadlib==1.9.8 lazr.restfulclient==0.11.2 lazr.uri==1.0.2 louis==2.3.0 lxml==2.3 mechanize==0.1.11 nvidia-common==0.0.0 oauth==1.0.1 onboard==0.96.1 oneconf==0.2.6.7 papyon==0.5.5 pexpect==2.3 piston-mini-client==0.6 protobuf==2.4.0a psycopg2==2.4.4 pyOpenSSL==0.12 pycrypto==2.3 pycups==1.9.59 pycurl==7.19.0 pyinotify==0.9.1 pyparsing==1.5.2 pyserial==2.5 pysmbc==1.0.10 python-apt==0.8.0ubuntu9 python-dateutil==1.4.1 python-debian==0.1.20ubuntu2 python-virtkey==0.60.0 pyxdg==0.19 sessioninstaller==0.0.0 simplejson==2.1.6 system-service==0.1.6 ubuntu-sso-client==1.4.0 ubuntuone-couch==0.3.0 ubuntuone-installer==2.0.0 ubuntuone-storage-protocol==2.0.0 ufw==0.30.1-2ubuntu1 unattended-upgrades==0.1 usb-creator==0.2.23 virtualenv==1.6.4 wadllib==1.2.0 wsgiref==0.1.2 xdiagnose==1.1 xkit==0.0.0 zope.interface==3.6.1 ``` When deploying on heroku it fails at Brlapi ..... I see lots of stuff from my main python installation which is on ubuntu. Which is BAD since Ubuntu use python for quite a few thing itself (ubuntu-one, usb-creator, etc..). I do not need them on heroku! I need only Django, psycopg2, and their dependencies. I do not even know if its fault of pip, or virutalenv. (If you want to know my setup look at link above I copied it into terminal)

Original source