====== Build Caffe container with singularity ====== In this section will show how to build ''caffe'' container using singularity. The goal is to show you how to build images on your machine, and copy them to the Mesocentre for execution. First, download install Singularity from here: http://singularity.lbl.gov. Once installed, follow the steps below: ===== Create the image ===== The image can be created with this commande: $ sudo create -size 2048 caffe-cpu.img This command will create a file named ''caffe-cpu.img'' of size 2Go. ===== Install or (bootsrap) the container the image ===== Next, we need install system (ubuntu), all required softwares and finally caffe. The Bootsrap file is given bellow: BootStrap: docker From: ubuntu:latest %setup # commands to be executed on host outside container during bootstrap %post # commands to be executed inside container during bootstrap # create bind points for MesoFC HPC environment mkdir -p /Work mkdir -p /Home ### Install depencies applications apt-get update && apt-get install -y --no-install-recommends \ build-essential \ cmake \ git \ wget \ libatlas-base-dev \ libboost-all-dev \ libgflags-dev \ libgoogle-glog-dev \ libhdf5-serial-dev \ libleveldb-dev \ liblmdb-dev \ libopencv-dev \ libprotobuf-dev \ libsnappy-dev \ protobuf-compiler \ python-dev \ python-numpy \ python-pip \ python-setuptools \ python-scipy && \ #rm -rf /var/lib/apt/lists/* ### Install caffe in /opt cd /opt git clone https://github.com/BVLC/caffe.git && \ cd caffe && pip install --upgrade pip && \ cd python && pip install -r requirements.txt&& \ cd ..&&mkdir build && cd build && \ cmake -DCPU_ONLY=1 .. && \ make -j"$(nproc)"&&make install ### update environement variable cd echo "PATH=/opt/caffe/build/install/bin:\$PATH" >> /environment echo "PYTHONPATH=/opt/caffe/build/install/python:\$PYTHONPATH" >> /environment echo "LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/opt/caffe/build/install/lib" >> /environment echo "LIBRARY_PATH=\$LIBRARY_PATH:/opt/caffe/build/install/lib" >> /environment echo "CPATH=\$CPATH:/opt/caffe/build/install/include" >> /environment echo "C_INCLUDE_PATH=\$C_INCLUDE_PATH:/opt/caffe/build/install/include" >> /environment echo "CPLUS_INCLUDE_PATH=\$CPLUS_INCLUDE_PATH:/opt/caffe/build/install/include" >> /environment echo "export PATH PYTHONPATH LD_LIBRARY_PATH LIBRARY_PATH CPATH C_INCLUDE_PATH CPLUS_INCLUDE_PATH" >> /environment ln -s /environment /.singularity.d/env/90-env.sh %runscript # commands to be executed when the container runs %test # commands to be executed within container at close of bootstrap process The command to use is: $ sudo singularity bootstrap caffe-cpu.img Singularity Where * caffe-cpu.img: the image created before * Singularity: the bootstrap file This will take some minutes (take a coffee). If all goes well, caffe is installed by success inside the container ''caffe-cpu.img'' in ''/opt/caffee'' folder. ===== Use the container ===== You can run caffe, in your host, inside the container: $ singularity exec caffe-cpu.img caffe ===== Use the build container on the Mesocentre ===== Just copy this file to your cluster account (''WORK'' folder) and run singularity as usual. For example for interactive usage: $ qlogin -lh_vmem=6G $ cd WORK $ module load tools/singularity $ singularity exec caffe-cpu.img caffe ====== Links ====== * [[caffe|Caffe Optimized for GPU]]