Python Environments
Page Contents
Virtual Environments
References:
- A non-magical introduction to Pip and Virtualenv for Python beginners by Jamie Matthews.
- Pipenv & Virtual Environments, Hitchhier's Guide to Python.
- Python Virtual Environments: A Primer
- Pipenv: Python Dev Workflow for Humans
Note: Recommend PipEnv - supersedes virtualenv - see next section!
VirtualEnv
# Install pip install virtualenv # Create project virtual env for Python 2 cd my_project_folder virtualenv my_project # virtualenv my_project will create a folder in the current # directory which will contain everything you need for this # private environment # Create for Python 3 python3 -m venv env # Now part of the distro and doesn't need virtualenv to be installed # Use the venv source my_project/bin/activate # ... Your prompt should become prefixed with your environment name # ... After activating, a new path to the Python interpretter in your # ... environment folder takes precedence so it will be used. Same for # ... the python package path lookup etc... # Install packages as usual pip install <package-name> # When finished... deactivate # Delete a venv rm -fr my_project # Snapshot of current state: pip freeze > requirements.txt # Load snapshot pip install -r requirements.txt
AutoEnv
You can supplement VirtualEnv with something called AutoEnv. It is a little utility that
will monitor your change-directory commands. When you cd
into a directory
that contains a .env
file it will try to source that file. So, if inside that
file you have the line source my_project/bin/activate
, when you change into
that directory, it will automatically activate your project for you. Happy days!
DirEnv
The author of AutoEnv actually recommends trying this above AutoEnv. TODO.
PyEnv
Install virtualenv: curl https://pyenv.run | bash
Add this to `~/.bashrc` and source:
export PYENV_ROOT="$HOME/.pyenv" command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH" eval "$(pyenv init -)" eval "$(pyenv virtualenv-init -)"PyEnv will try to build Python versions you install so add the following:
sudo apt update && sudo apt install -y \ build-essential \ checkinstall \ zlib1g-dev \ libbz2-dev \ libncurses5-dev \ libncursesw5-dev \ readline-common \ libreadline-dev \ libffi-dev \ libssl-dev \ liblzma-dev \ tk \ tk-dev \ libsqlite3-dev
To install a Python version in PyEnv:
pyenv update pyenv install 3.7:latest pyenv install 3.8:latest # Instruct Pyenv to first use system Python version, then versions # 3.7 and 3.8. pyenv global system 3.7.13 3.8.13
To create a virtual environment with PyEnv:
# Generally... pyenv virtualenv <python-version> <environment-name> # For example... pyenv virtualenv 3.7.4 my_new_project
To activate a virtual environment its:
pyenv activate <environment-name>
To automatically use an environment when in a development directory do:
cd ~/my/dev/directory pyenv local <my-dev-environment-name>
To upgrade PyEnv do
cd ~/.pyenv git pull
Pipenv - Python Virtual Environments and Package Management In One
See Pipenv: A Guide to the New Python Packaging Tool. See also Why Python devs should use Pipenv.
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world ...
... It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages ...
... The problems that Pipenv seeks to solve are multi-faceted:
- You no longer need to use pip and virtualenv separately. They work together.
- Managing a
requirements.txt
file can be problematic, so Pipenv uses Pipfile and Pipfile.lock to separate abstract dependency declarations from the last tested combination.- ...
... Pipfiles contain information for the dependencies of the project, and supersedes the requirements.txt file used in most Python projects.
Install using pip3 [--user] install pipenv
and create a new virtualenv
using pipenv --python 3[.minor]
.
Now, to add packages to your project use pipenv install <package>[~=maj.min[.patch]]
Like the original environments, it needs tobe actived: pipenv shell
.
Creates virtual environment if one doesn't exist. Note, pipenv creates all virtual
environments in a default location.
To push to production, lock your environment: pipenv lock
. Creates/updates
the Pipfile.lock
file (never edit this!). To install from your lock file
use pipenv install --ignore-pipfile
.
To leave a pipenv virtual environment just exit the current shell: pipenv starts a new shell session with the virtualenv path instead of changing the path in the current shell session.
Cheat Sheet
Command | Meaning |
---|---|
check | Checks for PyUp Safety security vulnerabilities and against PEP 508 markers provided in Pipfile. |
clean | Uninstalls all packages not specified in Pipfile.lock. |
graph | Displays currently-installed dependency graph information. |
install [--ignore-pipfile] | Installs provided packages and adds them to Pipfile, or (if no packages are given), installs all packages from Pipfile. Use ignore option to install from lock file. |
lock | Generates Pipfile.lock |
run | Spawns a command installed into the virtualenv. |
shell | Spawns a shell within the virtualenv. |
sync | Installs all packages specified in Pipfile.lock. |
update | Runs lock, then sync. |
Pipenv Competitors
There are a couple of competitors to pipenv that I've found at the time of writing (Aug 2021). The two that seem to pop up are Poetry and PDM. Pipenv is more mature than them, and Poetry is more mature than PDM.
Not sure that they give an amazing amount over pip + venv, especially given you have to install another tool and then maintain and keep that up to date. Perhaps for larger projects they are more useful or if you are publishing to PyPi?
Some rough notes comparing the various options:
venv + pip - Built in to Python 3 - Pip freeze does the dependency freezing recursively so you can restore *full* dependencies uasing a requirements.txt from pip freeze - dont need pipenv for this! - Updating all package dependencies - no built in command but there are one liners - e.g. https://stackoverflow.com/a/3452888/1517244. However these will not figure out the correct set of dependencies between all required packages to ensure they all work together - need likes of pipenv/poetry for that I think. - No PyUp integration but easy to pip install util for this - Needs multiple commands IIRC: need to env\Scripts\activate on window then pip install, pip freeze, pip install -r requirements.txt and so on, but this is hardly a hardship lol pipenv - venv and pip functionality under one command - Updating all packages to appropriate versions seems easier - Complains about caching speed - More mature than poetry - Can only spec one version of python - not able to do ranges. - liked built in hook for pyup - need more info about who curates DB driving it. poetry - Like pipenv but newer with better cache performance - Can do python >= and < (pipenv can't) - Dependency management seems more stringent than pipenv - small pedantry with ansi terminals but as --no-ansi option - install outside of pip to avoid any global dependency damage - nice - No PyUp integration but easy to pip install util for this PDM - Even faster in benchmarks - uses local packages - no venv at all! - uses same TOML file format and seems like it can coexist with poetry - **** Didn't work out the box for me on Windows :'( **** - Worked okay on WSL for me although I had to apt install an extra python package for virtual envs - python3-venv