Ceci est une ancienne révision du document !
Performance studies
The objective of this study is to analyze xgboost's performance on the cluster.
Prerequisites
- Python Ananoda 3.7
- Xgboost (Installed with conda)
- Mesoshared (CPU Xeon Platinum 8160, 2.1 GHZ) With 92 cores
How to
Covert the notebook to python script:
$ ipython nbconvert --to python 3.XGBoost
Results
Multithreaded support
Xgboost is multithreaed. We control the number of threads by
OMP_NUm_THREADS env variable
For example to for 8 cores:
$ module load lang/python/anaconda/3.7 $ OMP_NUM_TREADS=8 time ipython 3.XGBoost.py
| # Threads | Time(m) |
|---|---|
| 1 | 11m32s |
| 8 | 2m03s |
| 16 | 1m18s |
| 24 | 1m8s |
| 32 | 1m5s |
| 40 | 1m5s |
| 48 | 1m5s |
| 64 | 1m54s |
| 72 | 2m13s |
| 92 | 3m21s |
Conclusion : Uper limit scalability is about 24 cores
GPU support
We will use Volta GPU node.
The package
py-xgboost-gpu is installed in this node with conda
$ qlogin -qvolta.q -l h_vmem=20G