Ceci est une ancienne révision du document !


Performance studies

The objective of this study is to analyze xgboost's performance on the cluster.

Prerequisites

  • Python Ananoda 3.7
  • Xgboost (Installed with conda)
  • Mesoshared (CPU Xeon Platinum 8160, 2.1 GHZ) With 92 cores

Covert the notebook to python script:

$ ipython nbconvert --to python 3.XGBoost
Xgboost is multithreaed. We control the number of threads by OMP_NUm_THREADS env variable

For example to for 8 cores:

$ module load lang/python/anaconda/3.7
$ OMP_NUM_TREADS=8 time ipython 3.XGBoost.py
# Threads Time(m)
111m32s
82m03s
161m18s
241m8s
321m5s
401m5s
481m5s
641m54s
722m13s
923m21s
Conclusion : Uper limit scalability is about 24 cores

We will use Volta GPU node.

The package py-xgboost-gpu is installed in this node with conda
$ qlogin -qvolta.q -l h_vmem=20G
  • perf/xgboost.1562335712.txt.gz
  • Dernière modification : 2019/07/05 14:08
  • de admin