Différences

Ci-dessous, les différences entre deux révisions de la page.

Lien vers cette vue comparative

Les deux révisions précédentes Révision précédente
Prochaine révision
Révision précédente
tensorflow [2017/04/18 12:39] – admintensorflow [2019/01/29 14:34] (Version actuelle) – admin
Ligne 1: Ligne 1:
 ====== TensorFlow ====== ====== TensorFlow ======
 +<note warning>This documentation is for using tensorflow 1.4.1 and singularity 1.6.1 on K40 GPU node</note>
  
 TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them.
Ligne 8: Ligne 8:
 ====== How to use Tensorflow ====== ====== How to use Tensorflow ======
  
-TensorFlow **1.x** is available within Singularity container.+TensorFlow **1.x** is available within [[singularity| Singularity container]] 
 + 
 +====== How to install custom python package  ====== 
 +Singularity images are read only. But on can install custom python package outside the container. The packages will be installed in the HOME directory.  
 + 
 +Say we want to install ''panda'' package for example.  
 + 
 +We use ''mesoshared'' because the HOME directory is mounted in RW. 
 + 
 +Load singularity 
 +<code bash> 
 +$ export LC_ALL=C 
 +$ module load tools/singularity 
 +</code> 
 +Install  
 +<code bash> 
 +$ singularity exec /Softs/lumiere/tools/singularity/spool/tensorflow-gpu.img  pip install --user panda 
 +</code> 
 +Check installation 
 +<code bash> 
 +$ singularity exec /Softs/lumiere/tools/singularity/spool/tensorflow-gpu.img python  singularity exec /Softs/lumiere/tools/singularity/spool/tensorflow-gpu.img  python                   
 +Python 2.7.12 (default, Dec  4 2017, 14:50:18)  
 +[GCC 5.4.0 20160609] on linux2 
 +Type "help", "copyright", "credits" or "license" for more information. 
 +>>> import panda as pd 
 + 
 +</code> 
 + 
  
 ===== GPU version ===== ===== GPU version =====
  
-Here is an example of SGE script that use tensorFlow GPU version withing singularity.+We will use **tensorflow-gpu.img** container image.  
 + 
 +Here is an example of SGE script that use tensorFlow GPU version withing singularity:
  
 <code bash> <code bash>
Ligne 22: Ligne 52:
 #$ -N job_tensorflow #$ -N job_tensorflow
    
-i#$ -q tesla.q   ## we request tesla.q             +#$ -q tesla.q   ## we request tesla.q             
    
-#$ -l h_vmem=4G   ## we request 4G+#$ -l h_vmem=48G   ## tensorflow GPU requires lot of memory
    
  
Ligne 33: Ligne 63:
 ## run TF with singularity ## run TF with singularity
    
-singularity exec $SING_SPOOL/tensorflow-gpu.img python myTF_program.py+singularity exec --nv $SING_SPOOL/tensorflow-gpu.img python myTF_program.py
  
  
 </code>   </code>  
  
 +<note warning>Tensorflow GPU requires a lot of memory, you must ask h_vmem=48G as a minimum</note>
  
  
 ===== CPU version ===== ===== CPU version =====
 +
 +CPU version of TensorFlow is less powerful than the GPU version. 
 +
 +We will use Singularity container: **ubuntu-16.img**  
 +
 +<code bash>
 +
 +#!/bin/bash 
 +
 +#$ -o $JOB_NAME.o$JOB_ID
 +
 +#$ -N job_tensorflowCPU
 + 
 +#$ -q all.q   ## we request all.q             
 + 
 +#$ -l h_vmem=4G   ## we request 4G
 + 
 +
 +## load singularity
 + 
 +module load tools/singularity
 + 
 +## run TF within singularity
 + 
 +singularity exec $SING_SPOOL/ubuntu-16.img python myTF_program.py
 +
 +
 +</code>  
 + 
  • tensorflow.1492519198.txt.gz
  • Dernière modification : 2017/04/18 12:39
  • de admin