Table des matières

Miniconda within Singularity

Motivation

Because lumière cluster is based on an old operating system (Centos 6.10), GLIBC errors may occur when installing or using new conda packages. For example:

./python3: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by ./python3)
./python3: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by ./python3)
./python3: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by ./python3)''

Solution: the idea is to use a more recent version of GLIBC embedded in a singularity container.

advantage: access to the most recent version of Python and the majority of recent packages.

disadvantage: lack of conda activate and deactivate options.It is therefore impossible to use conda activate. command.

How to use conda within Singularity ?

We have created a wrapper around conda called conda_sing.

This new command will replace the classical conda command.

First load the dedicated environment :

$ module load miniconda/singularity
Since $HOME directory is read-only on compute nodes. You'll need to run the following command once to configure conda to use $WORK instead:
$ module load miniconda/singularity
$ meso_setup_conda 

Create new environment

Create a new environment called myEnv

$ conda_sing create -n myEnv

To list conda environments

$ conda_sing env list

Install new packages in the environment

Traditionally, the environment is activated to install the packages. This feature is not available with singularity.

To install package inside environment:

$ conda_sing install -n myEnv package1 package2

To list installed packages:

$ conda_sing list -n myEnv 

Run program in the environment

To run program using Singularity :

$ conda_sing run -n myEnv python myProgram.py

SGE script

anaconda.sge
#!/bin/bash
 
#$ -q all.q
 
#$ -l h_vmem=6G ## job will cancel if memory exceed
 
#$ -N Test_anaconda
 
module load miniconda/singularity
 
conda_sing run -n myEnv python myProgram.py