Differences between revisions 3 and 4
⇤ ← Revision 3 as of 2024-02-09 11:59:11 →
Size: 3067
Editor: stroth
Comment: Add example to set up a minimal jupyter notebook environment with micromamba
← Revision 4 as of 2024-02-09 12:00:46 → ⇥
Size: 3172
Editor: stroth
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:

== Example of a setup with conda ==
Line 14: Line 16:
This is a faster way to start working with a jupyter notebook.

How to setup and use Jupyter Notebook

Example of a setup with conda

Jupyter Notebook can easily be installed and started in a conda environment: 

conda create --name jupyternb notebook --channel conda-forge
conda activate jupyternb
jupyter notebook --no-browser --port 5998 --ip $(hostname -f)

  • After a successful start, the notebook prints the URL it's accessible under, which looks similar to
    http://<hostname>.ee.ethz.ch:5998/?token=5586e5faa082d5fe606efad0a0033ad0d6dd898fe0f5c7af

  • A notebook started like this in a Slurm job will be accessible

  • For access from a remote PC through VPN the port range should be [5900-5999]

Example of a minimal setup with micromamba

This is a faster way to start working with a jupyter notebook.

The following script sets up the small and fast conda package manager micromamba and creates a minimal conda environment to start a jupyter notebook: 

#!/bin/bash

# Set up micromamba
# Set up a python environment with $python_packages called $env_name

# Use net_scratch to install micromamba and python environments
# see https://computing.ee.ethz.ch/Services/NetScratch
MAMBA_ROOT_PREFIX="/usr/itetnas04/data-scratch-01/${USER}/data/micromamba"

micromamba_installer_url='https://micro.mamba.pm/api/micromamba/linux-64/latest'
env_name='jupyter_notebook'

# Minimal installation, takes ~1'
python_packages='notebook'
conda_channels='--channel conda-forge'

# Installation with pytorch and Cuda matching GPU driver in cluster:
# Takes more than 5'
#python_packages='notebook matplotlib scipy sqlite pytorch torchvision pytorch-cuda=11.8'
#conda_channels='--channel conda-forge --channel pytorch --channel nvidia'

mkdir -v -p "${MAMBA_ROOT_PREFIX}" &&
    cd "${MAMBA_ROOT_PREFIX}" &&
    # Download latest Micromamba (static linked binary)
    wget -O- "${micromamba_installer_url}" |
    # Extract Micromamba
    tar -xvj bin/micromamba &&
    # Set base path for Micromamba
    export MAMBA_ROOT_PREFIX &&
    # Initialize Micromamba
    eval "$(${MAMBA_ROOT_PREFIX}/bin/micromamba shell hook --shell=bash)" &&
    # Create the python environment for the course
    micromamba create --yes --name ${env_name} ${python_packages} ${conda_channels} &&

    # Show how to initialize micromamba
    echo 'Initialize micromamba immediately:' &&
    echo "    eval \"\$(${MAMBA_ROOT_PREFIX}/bin/micromamba shell hook --shell=bash)\"" &&
    echo &&
    echo "Activate environment ${env_name}:" &&
    echo "    micromamba activate ${env_name}"

  • Save this script as create_jupyter_env.sh

  • Make it executable with 

    chmod u+x ./create_jupyter_env.sh

  • And start it with ./create_jupyter_env.sh

After micromamba is installed and the environment is created, the script shows how to initialize micromamba and activate the environment.

After doing both, start a notebook as shown above.

FAQ/JupyterNotebook (last edited 2024-02-09 12:00:46 by stroth)