Afficher la pageAnciennes révisionsHaut de page Cette page est en lecture seule. Vous pouvez afficher le texte source, mais ne pourrez pas le modifier. Contactez votre administrateur si vous pensez qu'il s'agit d'une erreur. ====== Performance studies ====== The objective of this study is to analyze xgboost's performance on the cluster. **Prerequisites** * Python Ananoda 3.7 * Xgboost (Installed with conda) * Mesoshared (CPU Xeon Platinum 8160, 2.1 GHZ) With 92 cores ===== How to===== Covert the notebook to python script: <code bash> $ ipython nbconvert --to python 3.XGBoost </code> ===== Results ===== ==== Multithreaded support ==== <note tip>Xgboost is multithreaed. We control the number of threads by ''OMP_NUm_THREADS'' env variable </note> For example to for 8 cores: <code bash> $ module load lang/python/anaconda/3.7 $ OMP_NUM_TREADS=8 time ipython 3.XGBoost.py </code> ^# Threads ^Time(m)^ |1|11m32s| |8|2m03s| |16|1m18s| |24|1m8s| |32|1m5s| |40|1m5s| |**48**|**1m5s**| |64|1m54s| |72|2m13s| |92|3m21s| <note important>**Conclusion** : Uper limit scalability is about **24 cores**</note> ==== GPU support ==== We will use Volta GPU node. <note tip> The package ''py-xgboost-gpu'' is installed in this node with ''conda'' </note> <code bash> $ qlogin -qvolta.q -l h_vmem=20G </code> <note important>We need to modify our program to use GPU. I just added this param ''tree_method= 'gpu_hist''' in ''xgb_reg'' object.</note> <code bash> 150 xgb_reg = xgboost.XGBRegressor(learning_rate = 0.01, 151 max_depth = 10, 152 random_state=42, 153 n_estimators = 173, 154 tree_method= 'gpu_hist', 155 n_jobs=-1, 156 objective = 'count:poisson') </code> We load anaconda <code bash> $ export PATH=/opt/anaconda3/bin:$PATH </code> Finaly we run our program: <code> $ time ipython 3.XGBoostGPU.py </code> <color #ed1c24> **User time is about 38seconds**</color> <note tip>Using Multi GPU without sucess (Still reading docs :-) )</note> perf/xgboost.txt Dernière modification : 2019/07/05 14:58de admin