Virtualenv

virtualenv is software for isolating python package environments (e.g. import paths). It is written by Mozilla's own Ian Bicking. Creating a virtualenv installs setuptools (or optionally distribute) into the virtual environment. It is recommended that you use virtualenv for installing anything other that system software that you are hacking on. You will obviate the need to have access to your system packages and you won't mess up your system packages with whatever code you install into the site-packages directory of the virtualenv. site-packages should mostly be for system-installed python and not python that you are installing as a user.

This page is mostly a stub for how virtualenv is used at Mozilla. For more information on virtualenv in general, see the virtualenv web site. Python packaging at Mozilla is covered in Python.

Getting Virtualenv

virtualenv is a python package. You can download it from its PyPI package page or install it with easy_install. The github virtualenv repository is maintained by the Python Packaging Authority (including Mozilla's Tarek Ziade and Ian Bicking). If you have git installed, you can clone the latest version of virtualenv: git clone git://github.com/pypa/virtualenv.git

If you have pip or easy_install, you can install virtualenv directly from the web:

pip install virtualenv
# -or-
easy_install virtualenv

This will fetch virtualenv from PyPI and install it in your site-packages. You may need to use sudo to obtain appropriate permissions for the system. If you do not have pip or easy_install, you will need to download from PyPI or clone from the github virtualenv repository, extract from the tarball (in the PyPI case), and run python setup.py install. Again, you may need to use sudo to install globally. If you're running Ubuntu, there is also an Ubuntu package available:

sudo apt-get install python-virtualenv

virtualenv.py may also be run as standalone software with the same functionality. It will require it to be part of a clone of the github repository or have internet access to fetch setuptools.

On Mac OS X, you will need Xcode to use virtualenv. See this article for details.

Using Virtualenv

Once you have virtualenv installed, you can make virtual environments:

> virtualenv tmp
New python executable in tmp/bin/python
Installing setuptools............done.
Installing pip...............done.

A python binary, as well as easy_install and pip are available in the bin/ subdirectory (or Scripts on windows).

> ls tmp/bin/
activate      activate.fish     easy_install      pip      python
activate.csh  activate_this.py  easy_install-2.7  pip-2.7

Using this python binary, or these scripts (which point to this python binary), you will correctly install python packages in the lib/python2.x/site-packages directory and they will be appropriately added to your import path (sys.path) via lib/python2.x/site.py. In order to get the virtualenv's python binary on your PATH, you should source the bin/activate script on unix or run activate.bat on Windows.

Unix example

. bin/activate

Once the virtualenv is activated, the virtualenv's python (and other executables) will be on your PATH and you will have a new environment variable, VIRTUAL_ENV, that points to the path of the virtualenv, as well as a deactivate function for deactivating the virtualenv.

Virtualenv recipes

There are many different ways to use virtualenv in practice. One such recipe is as follows. Let's say you are working on mozbase and want to develop this in a virtualenv. Here are the steps:

Create a virtualenv: virtualenv mozbase
Activate the virtualenv: cd mozbase; . bin/activate
Create a source directory: mkdir src
Clone mozbase in the source directory: git clone [email protected]:mozilla/mozbase.git
Install packages in the virtualenv

For mozbase, we have a setup_development.py script which will install packages in the proper order. For a repository with a single package, just run python setup.py develop.

Note that this isn't the only way to use virtualenv, it is just one such recipe

Virtualenv tools

virtualenv does one thing well: creates isolated virtual environments of python packages. Due to its utility, tools have been built around this functionality. Here are a few such tools:

carton: make a self-extracting virtualenv from directories or URLs of packages; https://pypi.python.org/pypi/carton
velcro: a script that sets up a Python project for local installation; https://bitbucket.org/kumar303/velcro/
virtualenvwrapper: a set of extensions to Ian Bicking’s virtualenv tool for creating isolated Python development environments; https://www.doughellmann.com/projects...tualenvwrapper

The Mozilla-Central Virtualenv

In order to make use of various python modules located throughout mozilla-central, a virtualenv is created as part of the build process: https://mxr.mozilla.org/mozilla-central/source/js/src/build/autoconf/python-virtualenv.m4 . The virtualenv is created in ${OBJDIR}/_virtualenv and should be recreated as part of the configure step.

The populate_virtualenv.py script, when invoked, installs a list of packages, https://mxr.mozilla.org/mozilla-central/source/build/virtualenv/packages.txt , into the virtualenv via one of various methods. The Mozilla build system mostly uses .pth files instead of the more typically used python setup.py develop or python setup.py install to install python modules in to the virtualenv's python path due to performance concerns (although such functionality is available via the setup.py keyword in a packages.txt file). When using this method, be aware that the parts of package installation invoked via setup.py, such as console-script creation and dependency resolution, will not be invoked.

The virtualenv software, mirrored from a canonical version, lives in python/virtualenv.