Python

We provide some modules that come with the current Debian GNU/Linux stable release, but usually this is because they are dependencies of an installed software. For python we strongly recommend to build own python environments with the desired python versions and modules.

Our recommended way to install such environments is trough conda, expecially if you want to build a tool or toolchain where the setup will possibly be published in a paper. Alternatively, building an environment via pyenv is as possible.

For just quickly trying out some python tool a local installation of pip is recommended.

Installing your own python environment with Conda

For a detailed overview for conda please follow to the Conda documentation.

Installing your own python versions with pyenv

Pyenv is a collection of tools that allow users to manage different versions of python. In the simplest case you will need it to simply get an installation of python in your user space. Using that custom python installation, you will then be able to install additional modules in a very comfortable way, since you can install them in the "system path" (which is then somewhere within your home).

Documentation on pyenv can be found at https://github.com/pyenv/pyenv

Here is a small howto for installing python 3.9.1 in your home:

• Install pyenv:

  curl https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer -o pyenv-installer
  

  Check what the script is doing and then execute it:

  bash ./pyenv-installer
  

  You can remove the installer file afterwards.

• Add the following lines to your ~/.profile before sourcing ~/.bashrc`:

  export PYENV_ROOT="$HOME/.pyenv"
  export PATH="$PYENV_ROOT/bin:$PATH"
  eval "$(pyenv init --path)"
  

• In the ~/.bashrc:

  eval "$(pyenv init -)"
  

• If you want to pyenv-virtualenv automatically (in the ~/.bashrc):

  eval "$(pyenv virtualenv-init -)"
  

• You need a new login shell for all settings to take effect (when logged in on a Desktop environment logoff and login again)

• Install some python version, e.g. for python 3.9.1:

  env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.1
  pyenv rehash
  

  Note, that settting of PYTHON_CONFIGURE_OPTS="--enable-shared" is needed if you need to link against the libpython shared library.

• Make sure that this new python version will be used when you run python. You only need to run this command once:

  pyenv global 3.9.1
  

• In order to update pyenv run:

  pyenv update
  

Documentation of pyenv

• Website of pyenv	https://github.com/pyenv/pyenv
  Website of pyenv installer	https://github.com/pyenv/pyenv-installer

Installation of a local pip

pip can be installed in a user's environment and work with the python version installed on the system. Every module will be installed for the user only in one location, there is no separation with virtual environments.
Set up a local pip installation with the following commands:

mkdir -p ~/.local/bin
export PYTHONUSERBASE=~/.local
export PIP_USER=true
export PATH=$PYTHONUSERBASE/bin:$PATH
wget https://bootstrap.pypa.io/get-pip.py -O ~/.local/bin/get-pip.py
python3 ~/.local/bin/get-pip.py -vvv --user

Set default installations to the user's environment permanently (stored in ~/.config/pip/pip.conf):

pip config set install.user true

The exported environment variables will be lost after closing the shell. To enable local pip on demand, add the following function to your .bashrc:

function localpip {
    PYTHONUSERBASE=~/.local
    PATH=$PYTHONUSERBASE/bin:$PATH
    export PYTHONUSERBASE PATH
}

When you open a new shell, entering the command localpip will call the function and initialize your local pip installation.

Installation of additional or newer modules with pip

Once you installed your custom python with the explanations given above, you are ready to install additional or newer modules the easy way. The usage of pip is very easy. The following command installs the module numpy.

pip install numpy

while the next command would upgrade an existing installation of numpy

pip install --upgrade numpy

For advanced usage of pip, please consult the manuals: https://pip.pypa.io/en/latest/

pip cache

pip uses a cache which is by default stored under ~/.cache/pip or $XDG_CACHE_HOME/pip if it is set to a non-default location. This cache tends to fill up quickly and should occasionally be cleared with

pip cache purge

It is advisable to set the cache's location to the local scratch disk to avoid using up quota:

1. Create a directory for the cache:

   mkdir -p /scratch/$USER/pip_cache
   

2. Temporarily set the environment variable to tell pip to use a different cache location:

   export PIP_CACHE_DIR=/scratch/$USER/pip_cache/
   

   or store the location permanently (in ~/.config/pip/pip.conf):

   pip config set global.cache-dir /scratch/$USER/pip_cache
   

3. Check if the cache location is correct:

   pip cache info
   

Installation of Python modules that are not available in the archives of pip

Here we provide some shell script snippets for installing frequently asked modules which cannot be installed through pip. These scripts just provide an example installation. You might have to adapt some paths in order to make the module work correctly with the version of python you are using (e.g. if you run your custom python provided through pyenv).

nlopt

#!/bin/bash

# Installation script for nlopt library

VERSION=2.3
INSTALLDIR=$HOME/.local
BUILDDIR=/scratch/$USER/nlopt

mkdir -p $BUILDDIR
cd $BUILDDIR

wget "http://ab-initio.mit.edu/nlopt/nlopt-${VERSION}.tar.gz"
tar -xvvzkf nlopt-${VERSION}.tar.gz
cd nlopt-${VERSION}

./configure \
        --enable-shared \
        --prefix=$INSTALLDIR \
        OCT_INSTALL_DIR=$INSTALLDIR/octave/oct \
        M_INSTALL_DIR=$INSTALLDIR/octave/m/ \
        MEX_INSTALL_DIR=$INSTALLDIR/mex \
        GUILE_INSTALL_DIR=$INSTALLDIR/guile

make
make install

CategoryLXSW