Python

We provide some modules that come with the current Debian GNU/Linux stable release, but usually this is because they are dependencies of an installed software. For python we strongly recommend to build own python environments with the desired python versions and modules.

Our recommended way to install such environments is trough Conda, alternatively via building an environment via pyenv is as well possible. Additional module can then either be installed by conda itself or trough pip.

Installing your own python environment with Conda

For a detailed overview for conda please follow to the Conda documentation.

Installing your own python versions with pyenv

Pyenv is a collection of tools that allow users to manage different versions of python. In the simplest case you will need it to simply get an installation of python in your user space. Using that custom python installation, you will then be able to install additional modules in a very comfortable way, since you can install them in the "system path" (which is then somewhere within your home).

Documentation on pyenv can be found at https://github.com/pyenv/pyenv

Here is a small howto for installing python 3.9.1 in your home:

Install pyenv:

curl https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer -o pyenv-installer

Check what the script is doing and then execute it:

bash ./pyenv-installer

You can remove the installer file afterwards.

Add the following lines to your ~/.profile before sourcing ~/.bashrc`:

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"

In the ~/.bashrc:
```
eval "$(pyenv init -)"
```
If you want to pyenv-virtualenv automatically (in the ~/.bashrc):
```
eval "$(pyenv virtualenv-init -)"
```
You need a new login shell for all settings to take effect (when logged in on a Desktop environment logoff and login again)
Install some python version, e.g. for python 3.9.1:
```
env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.1
pyenv rehash
```
Note, that settting of PYTHON_CONFIGURE_OPTS="--enable-shared" is needed if you need to link against the libpython shared library.
Make sure that this new python version will be used when you run python. You only need to run this command once:
```
pyenv global 3.9.1
```
In order to update pyenv run:
```
pyenv update
```

Documentation of pyenv

Website of pyenv

https://github.com/pyenv/pyenv

Website of pyenv installer

https://github.com/pyenv/pyenv-installer

Installation of additional or newer modules with pip

Once you installed your custom python with the explanations given above, you are ready to install additional or newer modules the easy way. The usage of pip is very easy. The following command installs the module numpy.

pip install numpy

while the next command would upgrade an existing installation of numpy

pip install --upgrade numpy

For advanced usage of pip, please consult the manuals: https://pip.pypa.io/en/latest/

pip cache

pip uses a cache which is by default stored under ~/.cache/pip or $XDG_CACHE_HOME/pip if it is set to a non-default location. This cache tends to fill up quickly and should occasionally be cleared with

pip cache purge

It is advisable to set the cache's location to the local scratch disk to avoid using up quota:

Create a directory for the cache:
```
mkdir -p /scratch/$USER/pip_cache
```
Temporarily set the environment variable to tell pip to use a different cache location:
```
export PIP_CACHE_DIR=/scratch/$USER/tmp/
```
or create a configuration file to store the location permanently:
```
mkdir -p ~/.config/pip
cat << EOF > ~/.config/pip/pip.conf
[global]
cache-dir = /scratch/<my username>/pip_cache
EOF
```
Make sure to replace <my username> with your actual username in pip.conf
Check if the cache location is correct:
```
pip cache info
```

Installation of Python modules that are not available in the archives of pip

Here we provide some shell script snippets for installing frequently asked modules which cannot be installed through pip. These scripts just provide an example installation. You might have to adapt some paths in order to make the module work correctly with the version of python you are using (e.g. if you run your custom python provided through pyenv).

nlopt

   1 #!/bin/bash
   2 
   3 # Installation script for nlopt library
   4 
   5 VERSION=2.3
   6 INSTALLDIR=$HOME/.local
   7 BUILDDIR=/scratch/$USER/nlopt
   8 
   9 mkdir -p $BUILDDIR
  10 cd $BUILDDIR
  11 
  12 wget "http://ab-initio.mit.edu/nlopt/nlopt-${VERSION}.tar.gz"
  13 tar -xvvzkf nlopt-${VERSION}.tar.gz
  14 cd nlopt-${VERSION}
  15 
  16 ./configure \
  17         --enable-shared \
  18         --prefix=$INSTALLDIR \
  19         OCT_INSTALL_DIR=$INSTALLDIR/octave/oct \
  20         M_INSTALL_DIR=$INSTALLDIR/octave/m/ \
  21         MEX_INSTALL_DIR=$INSTALLDIR/mex \
  22         GUILE_INSTALL_DIR=$INSTALLDIR/guile
  23 
  24 make
  25 make install

CategoryLXSW

-  ⇤ ← Revision 42 as of 2016-04-20 05:30:58 → 
  Size: 3998
  Editor: bonaccos
  Comment:
+   ← Revision 62 as of 2023-03-31 07:28:15 → ⇥
  Size: 5484
  Editor: stroth
  Comment: Add info about relocation/purging pip cache
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
+#rev 2020-09-10 bonaccos

<<TableOfContents()>>
-Line 3:
+Line 7:
-We provide as many modules as possible that come with the current Debian GNU/Linux stable release. Nevertheless, that might not be enough for your needs since you may want to use the newest version of some module or one that is not part of Debian. Furthermore you might want to use a different version of Python.
+We provide some modules that come with the current Debian GNU/Linux stable release, but usually this is because they are dependencies of an installed software. For python we strongly recommend to build own python environments with the desired python versions and modules.
-Line 5:
+Line 9:
-The recommended way to install additional python modules is through {{{pip}}}. Unfortunately, {{{pip}}} does not allow users to install modules in the user context. That means that you will first have to install your own version of python in your home. From that moment on, you can install modules through {{{pip}}}.
+Our recommended way to install such environments is trough Conda, alternatively via building an environment via `pyenv` is as well possible. Additional module can then either be installed by conda itself or trough `pip`.
-Line 7:
+Line 11:
+== Installing your own python environment with Conda ==

For a detailed overview for conda please follow to the [[Programming/Languages/Conda|Conda documentation]].
-Line 10:
+Line 17:
-{{{Pyenv}}} is a collection of tools that allow users to manage different versions of python. In the simplest case you will need it to simply get an installation of python in your user space. Using that custom python installation, you will then be able to install additional modules in a very comfortable way, since you can install them in the "system path" (which is then somewhere within your home).
+`Pyenv` is a collection of tools that allow users to manage different versions of python. In the simplest case you will need it to simply get an installation of python in your user space. Using that custom python installation, you will then be able to install additional modules in a very comfortable way, since you can install them in the "system path" (which is then somewhere within your home).
-Line 12:
+Line 19:
-Here is a small howto for installing python 2.7.11 in your home:
+Documentation on `pyenv` can be found at https://github.com/pyenv/pyenv
-Line 14:
+Line 21:
- * Install pyenv:
+Here is a small howto for installing python 3.9.1 in your home:

 * Install pyenv:
-Line 16:
+Line 25:
-curl https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer -o pyenv-installer}}}
+curl https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer -o pyenv-installer
}}}
-Line 22:
+Line 32:
- * Add the following three lines to your ~/.bashrc:
+ * Add the following lines to your `~/.profile` before sourcing ~/.bashrc`:
-Line 24:
+Line 34:
-export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)" 
eval "$(pyenv virtualenv-init -)"}}}
+export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"
}}}
 * In the `~/.bashrc`:
 {{{
eval "$(pyenv init -)"
}}}
 * If you want to pyenv-virtualenv automatically (in the `~/.bashrc`):
 {{{
eval "$(pyenv virtualenv-init -)"
}}}
-Line 28:
+Line 47:
- * Restart your shell so the path changes take effect:
+ * You need a new login shell for all settings to take effect (when logged in on a Desktop environment logoff and login again)

 * Install some python version, e.g. for python 3.9.1:
-Line 30:
+Line 51:
-exec $SHELL}}}

 * Install some python version, e.g. for python 2.7.11:
 {{{
env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 2.7.11
+env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.1
-Line 36:
+Line 53:
- Note, that settting of `PYTHON_CONFIGURE_OPTS="--enable-shared"` is needef if you need to link against the libpython shared library.
+ Note, that settting of `PYTHON_CONFIGURE_OPTS="--enable-shared"` is needed if you need to link against the libpython shared library.
-Line 40:
+Line 57:
-pyenv global 2.7.11}}}
+pyenv global 3.9.1}}}
-Line 47:
+Line 64:
- || Website of pyenv || https://github.com/yyuu/pyenv/ ||
 || Website of pyenv installer || https://github.com/yyuu/pyenv-installer ||
+ || Website of pyenv || https://github.com/pyenv/pyenv ||
 || Website of pyenv installer || https://github.com/pyenv/pyenv-installer ||
-Line 53:
+Line 70:
-Once you installed your custom python with the explanations given above, you are ready to install additional or newer modules the easy way. The usage of {{{pip}}} is very easy. The following command installs the module {{{numpy}}}
+Once you installed your custom python with the explanations given above, you are ready to install additional or newer modules the easy way. The usage of `pip` is very easy. The following command installs the module `numpy`.
-Line 55:
+Line 72:
-pip install numpy}}}
+pip install numpy
}}}
-Line 58:
+Line 76:
-pip install --upgrade numpy}}}
+pip install --upgrade numpy
}}}
-Line 60:
+Line 79:
-For advanced usage of {{{pip}}}, please consult the manuals: http://pip.readthedocs.org/en/latest/
+For advanced usage of `pip`, please consult the manuals: https://pip.pypa.io/en/latest/

=== pip cache ===
`pip` uses a cache which is by default stored under `~/.cache/pip` or `$XDG_CACHE_HOME/pip` if it is set to a non-default location. This cache tends to fill up quickly and should occasionally be cleared with
{{{#!highlight bash numbers=disable
pip cache purge
}}}
It is advisable to set the cache's location to the local scratch disk to avoid using up quota:

 1. Create a directory for the cache: {{{#!highlight bash numbers=disable
mkdir -p /scratch/$USER/pip_cache
}}}
 1. Temporarily set the environment variable to tell `pip` to use a different cache location: {{{#!highlight bash numbers=disable
export PIP_CACHE_DIR=/scratch/$USER/tmp/
}}}
 1. or create a configuration file to store the location permanently: {{{#!highlight bash numbers=disable
mkdir -p ~/.config/pip
cat << EOF > ~/.config/pip/pip.conf
[global]
cache-dir = /scratch/<my username>/pip_cache
EOF
}}} Make sure to replace `<my username>` with your actual username in `pip.conf`
 1. Check if the cache location is correct: {{{#!highlight bash numbers=disable
pip cache info
}}}
-Line 65:
+Line 108:
-Here we provide some shell script snippets for installing frequently asked modules which cannot be installed through {{{pip}}}. These scripts just provide an example installation. You might have to adapt some paths in order to make the module work correctly with the version of python you are using (e.g. if you run your custom python provided through {{{pyenv}}}).
+Here we provide some shell script snippets for installing frequently asked modules which cannot be installed through {{{pip}}}. These scripts just provide an example installation. You might have to adapt some paths in order to make the module work correctly with the version of python you are using (e.g. if you run your custom python provided through `pyenv`).

Wiki

Page