Table des matières

Build Caffe container with singularity

In this section will show how to build caffe container using singularity. The goal is to show you how to build images on your machine, and copy them to the Mesocentre for execution.

First, download install Singularity from here: http://singularity.lbl.gov.

Once installed, follow the steps below:

Create the image

The image can be created with this commande:

$ sudo create -size 2048 caffe-cpu.img

This command will create a file named caffe-cpu.img of size 2Go.

Install or (bootsrap) the container the image

Next, we need install system (ubuntu), all required softwares and finally caffe.

The Bootsrap file is given bellow:

Singularity
BootStrap: docker
From: ubuntu:latest
 
%setup
# commands to be executed on host outside container during bootstrap
 
%post
# commands to be executed inside container during bootstrap
 
# create bind points for MesoFC HPC environment
mkdir -p /Work
mkdir -p /Home
 
### Install depencies applications
apt-get update && apt-get install -y --no-install-recommends \
        build-essential \
        cmake \
        git \
        wget \
        libatlas-base-dev \
        libboost-all-dev \
        libgflags-dev \
        libgoogle-glog-dev \
        libhdf5-serial-dev \
        libleveldb-dev \
        liblmdb-dev \
        libopencv-dev \
        libprotobuf-dev \
        libsnappy-dev \
        protobuf-compiler \
        python-dev \
        python-numpy \
        python-pip \
        python-setuptools \
        python-scipy && \
#rm -rf /var/lib/apt/lists/*
 
### Install caffe in /opt
cd /opt
 
git clone  https://github.com/BVLC/caffe.git  && \
    cd caffe && pip install --upgrade pip && \
    cd python && pip install -r requirements.txt&& \
    cd ..&&mkdir build && cd build && \
    cmake -DCPU_ONLY=1 .. && \
    make -j"$(nproc)"&&make install
 
 
### update environement variable
cd 
echo "PATH=/opt/caffe/build/install/bin:\$PATH" >> /environment
echo "PYTHONPATH=/opt/caffe/build/install/python:\$PYTHONPATH" >> /environment 
echo "LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/opt/caffe/build/install/lib" >> /environment 
echo "LIBRARY_PATH=\$LIBRARY_PATH:/opt/caffe/build/install/lib" >> /environment 
echo "CPATH=\$CPATH:/opt/caffe/build/install/include" >> /environment  
echo "C_INCLUDE_PATH=\$C_INCLUDE_PATH:/opt/caffe/build/install/include" >>  /environment 
echo "CPLUS_INCLUDE_PATH=\$CPLUS_INCLUDE_PATH:/opt/caffe/build/install/include" >> /environment 
 
echo "export PATH PYTHONPATH LD_LIBRARY_PATH LIBRARY_PATH CPATH C_INCLUDE_PATH CPLUS_INCLUDE_PATH" >> /environment 
ln -s /environment /.singularity.d/env/90-env.sh
 
%runscript
# commands to be executed when the container runs
 
%test
# commands to be executed within container at close of bootstrap process

The command to use is:

$ sudo singularity bootstrap caffe-cpu.img Singularity 

Where

This will take some minutes (take a coffee).

If all goes well, caffe is installed by success inside the container caffe-cpu.img in /opt/caffee folder.

Use the container

You can run caffe, in your host, inside the container:

$ singularity exec caffe-cpu.img caffe

Use the build container on the Mesocentre

Just copy this file to your cluster account (WORK folder) and run singularity as usual.

For example for interactive usage:

$ qlogin -lh_vmem=6G
$ cd WORK
$ module load tools/singularity
$ singularity exec caffe-cpu.img caffe 

Links