Différences

Ci-dessous, les différences entre deux révisions de la page.

Lien vers cette vue comparative

Les deux révisions précédentes Révision précédente
Prochaine révision
Révision précédente
anaconda_sing [2023/12/07 18:56] – adminanaconda_sing [2023/12/08 08:29] (Version actuelle) – [Motivation] admin
Ligne 2: Ligne 2:
  
 ===== Motivation ===== ===== Motivation =====
-Because lumière cluster is based on an old operating system (Centos 6.10), GLIBC errors may occur when installing or using conda packages.+Because lumière cluster is based on an old operating system (''Centos 6.10''), GLIBC errors may occur when installing or using new conda packages. For example:
  
-The idea is to use a more recent version of GLIBC embedded in a singularity container.  
  
-**advantage: access to the most recent version of Python and the majority of recent packages.**+<code> 
 +./python3: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by ./python3) 
 +./python3: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by ./python3) 
 +./python3: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by ./python3)'' 
 +</code> 
 +----
  
-<note important><color #ed1c24>disadvantage: lack of conda activate and deactivate options. It is therefore impossible to use ''conda activate''.</color></note>  
  
-===== How to use  Singularity ? =====+ 
 + 
 + 
 +__**Solution:** the idea is to use a more recent version of ''GLIBC'' embedded in a singularity container.__  
 + 
 +<color #22b14c>**advantage: access to the most recent version of Python and the majority of recent packages.**</color> 
 + 
 +**disadvantage: lack of conda activate and deactivate options.<color #ed1c24>It is therefore impossible to use ''conda activate''.</color>** command. 
 + 
 +===== How to use  conda within Singularity ? =====
  
 We have created a wrapper around conda called **conda_sing**.\\ We have created a wrapper around conda called **conda_sing**.\\
Ligne 21: Ligne 33:
 $ module load miniconda/singularity $ module load miniconda/singularity
 </code> </code>
- + 
 +<note important> 
 +Since ''$HOME'' directory is read-only on compute nodes. You'll need to run the following command __**once**__ to configure ''conda'' to use ''$WORK'' instead: 
 +<code bash> 
 +$ module load miniconda/singularity 
 +$ meso_setup_conda  
 +</code> 
 +</note> 
 +==== Create new environment ==== 
 + 
 +Create a new environment called ''myEnv'' 
 + 
 +<code bash> 
 +$ conda_sing create -n myEnv 
 +</code> 
 + 
 + 
 +To list conda environments 
 +<code bash> 
 +$ conda_sing env list 
 +</code> 
 + 
 + 
 + 
 +==== Install new packages in the environment ==== 
 +<note warning> 
 +Traditionally, the environment is activated to install the packages. This feature is not available with singularity. 
 +</note> 
 + 
 +To install package inside environment: 
 + 
 +<code bash> 
 +$ conda_sing install -n myEnv package1 package2 
 +</code>  
 + 
 + 
 +To list installed packages: 
 +<code bash> 
 +$ conda_sing list -n myEnv  
 +</code> 
 +==== Run program in the environment ==== 
 +To run program using Singularity : 
 + 
 +<code bash> 
 +$ conda_sing run -n myEnv python myProgram.py 
 +</code>  
 + 
 + 
 +==== SGE script ==== 
 + 
 +<file bash anaconda.sge> 
 + 
 +#!/bin/bash 
 + 
 +#$ -q all.q 
 + 
 +#$ -l h_vmem=6G ## job will cancel if memory exceed 
 + 
 +#$ -N Test_anaconda 
 + 
 +module load miniconda/singularity 
 + 
 +conda_sing run -n myEnv python myProgram.py 
 + 
 + 
 +</file>  
 + 
  • anaconda_sing.1701975370.txt.gz
  • Dernière modification : 2023/12/07 18:56
  • de admin