====== Performance studies ======
The objective of this study is to analyze xgboost's performance on the cluster.
**Prerequisites**
* Python Ananoda 3.7
* Xgboost (Installed with conda)
* Mesoshared (CPU Xeon Platinum 8160, 2.1 GHZ) With 92 cores
===== How to=====
Covert the notebook to python script:
$ ipython nbconvert --to python 3.XGBoost
===== Results =====
==== Multithreaded support ====
Xgboost is multithreaed. We control the number of threads by ''OMP_NUm_THREADS'' env variable
For example to for 8 cores:
$ module load lang/python/anaconda/3.7
$ OMP_NUM_TREADS=8 time ipython 3.XGBoost.py
^# Threads ^Time(m)^
|1|11m32s|
|8|2m03s|
|16|1m18s|
|24|1m8s|
|32|1m5s|
|40|1m5s|
|**48**|**1m5s**|
|64|1m54s|
|72|2m13s|
|92|3m21s|
**Conclusion** : Uper limit scalability is about **24 cores**
==== GPU support ====
We will use Volta GPU node.
The package ''py-xgboost-gpu'' is installed in this node with ''conda''
$ qlogin -qvolta.q -l h_vmem=20G
We need to modify our program to use GPU.
I just added this param ''tree_method= 'gpu_hist''' in ''xgb_reg'' object.
150 xgb_reg = xgboost.XGBRegressor(learning_rate = 0.01,
151 max_depth = 10,
152 random_state=42,
153 n_estimators = 173,
154 tree_method= 'gpu_hist',
155 n_jobs=-1,
156 objective = 'count:poisson')
We load anaconda
$ export PATH=/opt/anaconda3/bin:$PATH
Finaly we run our program:
$ time ipython 3.XGBoostGPU.py
**User time is about 38seconds**
Using Multi GPU without sucess (Still reading docs :-) )