====== TensorFlow ======
This documentation is for using tensorflow 1.4.1 and singularity 1.6.1 on K40 GPU node
TensorFlowâ„¢ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them.
====== How to use Tensorflow ======
TensorFlow **1.x** is available within [[singularity| Singularity container]]
====== How to install custom python package ======
Singularity images are read only. But on can install custom python package outside the container. The packages will be installed in the HOME directory.
Say we want to install ''panda'' package for example.
We use ''mesoshared'' because the HOME directory is mounted in RW.
Load singularity
$ export LC_ALL=C
$ module load tools/singularity
Install
$ singularity exec /Softs/lumiere/tools/singularity/spool/tensorflow-gpu.img pip install --user panda
Check installation
$ singularity exec /Softs/lumiere/tools/singularity/spool/tensorflow-gpu.img python singularity exec /Softs/lumiere/tools/singularity/spool/tensorflow-gpu.img python
Python 2.7.12 (default, Dec 4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import panda as pd
===== GPU version =====
We will use **tensorflow-gpu.img** container image.
Here is an example of SGE script that use tensorFlow GPU version withing singularity:
#!/bin/bash
#$ -o $JOB_NAME.o$JOB_ID
#$ -N job_tensorflow
#$ -q tesla.q ## we request tesla.q
#$ -l h_vmem=48G ## tensorflow GPU requires lot of memory
## load singularity
module load tools/singularity
## run TF with singularity
singularity exec --nv $SING_SPOOL/tensorflow-gpu.img python myTF_program.py
Tensorflow GPU requires a lot of memory, you must ask h_vmem=48G as a minimum
===== CPU version =====
CPU version of TensorFlow is less powerful than the GPU version.
We will use Singularity container: **ubuntu-16.img**
#!/bin/bash
#$ -o $JOB_NAME.o$JOB_ID
#$ -N job_tensorflowCPU
#$ -q all.q ## we request all.q
#$ -l h_vmem=4G ## we request 4G
## load singularity
module load tools/singularity
## run TF within singularity
singularity exec $SING_SPOOL/ubuntu-16.img python myTF_program.py