利用caffe的python接口进行训练时遇到的问题
2016-08-16 14:16
525 查看
利用caffe的python接口来训练:
然而,继续运行train.py,并没有输出梯度,找了很久的原因才发现,原来_caffe.so已经被安装到系统路径中,即‘/usr/lib/python2.7/dist-packages/caffe’中,所以在运行train.py的时候无论你对caffe的源码做了什么样的修改,都会默认调用系统路径里的库。所以,需要将/usr/lib/python2.7/dist-packages 中的caffe库全部删掉,然后用PYTHONPATH指定路径。
1 进入/usr/lib/python2.7/dist-packages , ls
包含caffe。
2 进入/usr/lib/python2.7/dist-packages/caffe, lld _caffe.so
3 删除
4 编辑 .bashrc,添加PYTHONPATH环境变量
然后
然后
python ./train.py, 想输出梯度,便在solver.cpp里做了修改,添加代码:
int max_len = 0; for (int l = 0; l < net_->layers().size(); ++l) { Layer<Dtype>& layer = *net_->layers()[l].get(); if (layer.layer_param().name().length() > max_len) max_len = layer.layer_param().name().length(); } // bottom blobs and their gradient const vector<vector<Blob<Dtype>*> >& bottom_vecs = net_->bottom_vecs(); for (int l = 0; l < net_->layers().size(); ++l) { Layer<Dtype>& layer = *net_->layers()[l].get(); for (int b=0; b<bottom_vecs[l].size(); ++b) { const Blob<Dtype>* blob = bottom_vecs[l][b]; Dtype data_sum = 0; Dtype diff_sum = 0; data_sum = blob->asum_data(); diff_sum = blob->asum_diff(); data_sum /= blob->count(); diff_sum /= blob->count(); LOG_IF(INFO, Caffe::root_solver()) << std::left << std::setw(max_len + 1) << std::setfill(' ') << layer.layer_param().name() << " bottom_blob " << b << ": " << std::scientific << data_sum << " [" << diff_sum << "]"; } } LOG(INFO) << std::endl; // output learnable weights and their gradients for (int l = 0; l < net_->layers().size(); ++l) { Layer<Dtype>& layer = *net_->layers()[l].get(); for (int bp = 0; bp < layer.blobs().size(); ++bp) { Blob<Dtype>& para_blob = *layer.blobs()[bp].get(); const Dtype* para_blob_cpu_data = para_blob.cpu_data(); const Dtype* para_blob_cpu_diff = para_blob.cpu_diff(); Dtype para_data_sum = 0; Dtype para_diff_sum = 0; for (int i = 0; i < para_blob.count(); ++i) { para_data_sum += (para_blob_cpu_data[i] > Dtype(0.)) ? para_blob_cpu_data[i] : - para_blob_cpu_data[i]; para_diff_sum += (para_blob_cpu_diff[i] > Dtype(0.)) ? para_blob_cpu_diff[i] : - para_blob_cpu_diff[i]; } para_data_sum /= para_blob.count(); para_diff_sum /= para_blob.count(); LOG_IF(INFO, Caffe::root_solver()) << std::left << std::setw(max_len + 1) << std::setfill(' ') << layer.layer_param().name() << " param_blob " << bp << ": " << std::scientific << para_data_sum << " [" << para_diff_sum << "]"; } } // end add by Zzp
然而,继续运行train.py,并没有输出梯度,找了很久的原因才发现,原来_caffe.so已经被安装到系统路径中,即‘/usr/lib/python2.7/dist-packages/caffe’中,所以在运行train.py的时候无论你对caffe的源码做了什么样的修改,都会默认调用系统路径里的库。所以,需要将/usr/lib/python2.7/dist-packages 中的caffe库全部删掉,然后用PYTHONPATH指定路径。
1 进入/usr/lib/python2.7/dist-packages , ls
mini@mini:/usr/lib/python2.7/dist-packages$ ls adium_theme_ubuntu-0.3.4.egg-info PAM.x86_64-linux-gnu.so ANSI.py pandas ANSI.pyc pandas-0.13.1.egg-info antlr.py patsy antlr.pyc patsy-0.2.1.egg-info apt pcardext.la aptdaemon pcardext.so apt_inst.so pep8-1.4.6.egg-info apt_pkg.so pep8.py aptsources pep8.pyc apt_xapian_index-0.45.egg-info pexpect astroid pexpect-3.1.egg-info astroid-1.0.1.egg-info PIL axi PILcompat beautifulsoup4-4.2.1.egg-info PILcompat.pth blinker Pillow-2.3.0.egg-info blinker-1.3.egg-info piston_mini_client boost piston_mini_client-0.7.5.egg-info bs4 pkg_resources.py caffe pkg_resources.pyc cairo protobuf-2.5.0.egg-info chardet psutil chardet-2.0.1.egg-info psutil-1.2.1.egg-info CommandNotFound _psutil_linux.x86_64-linux-gnu.so command_not_found-0.3.egg-info _psutil_posix.x86_64-linux-gnu.so Crypto pxssh.py cupsext.la pxssh.pyc cupsext.so pycrypto-2.6.1.egg-info cupshelpers pycups-1.9.66.egg-info cups.so pydot-1.0.28.egg-info cvxopt pydot.py cvxopt-1.1.4.egg-info pydot.pyc Cython pyflakes Cython-0.20.1post0.egg-info pyflakes-0.8.1.egg-info cython.py pygments cython.pyc Pygments-1.6.egg-info dateutil pygobject-3.12.0.egg-info dbus pygtkcompat _dbus_bindings.so pygtk.pth _dbus_glib_bindings.so pygtk.py deb822.py pygtk.pyc deb822.pyc pyinotify-0.9.4.egg-info debconf.py pyinotify.py debconf.pyc pyinotify.pyc debian pylint debian_bundle pylint-1.1.0.egg-info debtagshw pyOpenSSL-0.13.egg-info debtagshw-0.1.egg-info pyparsing-2.0.1.egg-info decorator-3.4.0.egg-info pyparsing.py decorator.py pyparsing.pyc decorator.pyc PyQt4 defer pyserial-2.6.egg-info defer-1.0.6.egg-info pysmbc-1.0.14.1.egg-info dirspec pysvn dirspec-13.10.egg-info python_apt-0.9.3.5ubuntu2.egg-info docutils python_dateutil-1.5.egg-info docutils-0.11.egg-info python_debian-0.1.21_nmu2ubuntu2.egg-info dot_parser.py python_gflags-1.5.1.egg-info dot_parser.pyc python-support.pth drv_libxml2.py pytz drv_libxml2.pyc pytz-2012c.egg-info duplicity pyxdg-0.25.egg-info duplicity-0.6.23.egg-info pyximport easy_install.py PyYAML-3.10.egg-info easy_install.pyc pyzmq-14.0.1.egg-info fdpexpect.py README fdpexpect.pyc reportlab flask reportlab-3.0.egg-info Flask-0.10.1.egg-info requests flaskext requests-2.2.1.egg-info flask_socketio roman-2.0.0.egg-info Flask_SocketIO-0.6.0.egg-info roman.py flask_wtf roman.pyc Flask_WTF-0.11.egg-info rope FSM.py rope-0.9.2.egg-info FSM.pyc samba gevent scanext.la gevent-1.0.egg-info scanext.so gevent_socketio-0.3.6.egg-info scikit_image-0.9.3.egg-info geventwebsocket scikit_learn-0.14.1.egg-info gevent_websocket-0.9.3.egg-info scipy gflags.py scipy-0.13.3.egg-info gflags.pyc screen.py gflags_validators.py screen.pyc gflags_validators.pyc serial gi sessioninstaller glib sessioninstaller-0.0.0.egg-info gobject setuptools google setuptools-3.3.egg-info greenlet-0.4.2.egg-info setuptools.pth greenlet.so simplegeneric-0.8.1.egg-info gtk-2.0 simplegeneric.py gtk-2.0-pysupport-compat.pth simplegeneric.pyc gunicorn simplejson gunicorn-17.5.egg-info simplejson-3.3.1.egg-info gyp sipconfig_nd.py gyp-0.1.egg-info sipconfig_nd.pyc h5py sipconfig.py h5py-2.2.1.egg-info sipconfig.pyc hgext sip.so hpmudext.la six-1.5.2.egg-info hpmudext.so six.py html5lib six.pyc html5lib-0.999.egg-info skimage httplib2 sklearn httplib2-0.8.egg-info smbc ibus _smbc.so indicator_keyboard socketio IPython softwarecenter_aptd_plugins ipython-1.2.1.egg-info software_center_aptd_plugins-0.0.0.egg-info itsdangerous-0.22.egg-info sphinx itsdangerous.py Sphinx-1.2.2.egg-info itsdangerous.pyc spyder-2.2.5.egg-info jinja2 spyderlib Jinja2-2.7.2.egg-info spyderplugins ldb.so ssh_import_id-3.21.egg-info _ldb_text.py statsmodels _ldb_text.pyc statsmodels-0.5.0.egg-info leveldb-0.1.egg-info system_service-0.1.6.egg-info leveldb.so tables libxml2mod.so tables-3.1.1.egg-info libxml2.py talloc.so libxml2.pyc tdb.so lmdb _tdb_text.py lmdb-0.87.egg-info _tdb_text.pyc lockfile-0.8.egg-info twisted lockfile.py Twisted_Core-13.2.0.egg-info lockfile.pyc Twisted_Web-13.2.0.egg-info logilab ubuntu-sso-client logilab_common-0.61.0.egg-info ubuntu-sso-client.pth lsb_release.py UbuntuSystemService lsb_release.pyc unity_lens_photos-1.0.egg-info lxml urllib3 lxml-3.3.3.egg-info urllib3-1.7.1.egg-info _markerlib werkzeug markupsafe Werkzeug-0.9.4.egg-info MarkupSafe-0.18.egg-info wtforms mercurial WTForms-2.0.1.egg-info mercurial-2.8.2.egg-info xapian mx xdg networkx xdiagnose networkx-1.8.1.egg-info xdiagnose-3.6.3build2.egg-info nose xlrd nose-1.3.1.egg-info xlrd-0.9.2.egg-info numexpr xlwt numexpr-2.2.2.egg-info xlwt-0.7.5.egg-info numpy yaml numpy-1.8.2.egg-info _yaml.so oauthlib zeitgeist oauthlib-0.6.1.egg-info zmq oneconf zope oneconf-0.3.7.14.04.1.egg-info zope.interface-4.0.5.egg-info OpenSSL zope.interface-4.0.5-nspkg.pth
包含caffe。
2 进入/usr/lib/python2.7/dist-packages/caffe, lld _caffe.so
mini@min:/usr/lib/python2.7/dist-packages/caffe$ ldd _caffe.so | grep caffe libcaffe-nv.so.0.14 => /usr/lib/x86_64-linux-gnu/libcaffe-nv.so.0.14 (0x00007f655b84b000) #libcaffe被安装到了系统路径,所以在运行train.py的时候无论你对caffe的源码做了什么样的修改,都会默认调用系统路径里的库
3 删除
mini@mini:/usr/lib/python2.7/dist-packages$ sudo rm -rf caffe
4 编辑 .bashrc,添加PYTHONPATH环境变量
export PYTHONPATH='/home/mm/SoftWare/caffe/python',
然后
..bashrc,生效
然后
popd
相关文章推荐
- caffe--python版利用训练好模型进行测试
- [原]有关Caffe训练好的模型在Python接口下使用分类不准确的问题解决
- 有关Caffe训练好的模型在Python接口下使用分类不准确的问题解决
- 【Caffe的C++接口使用说明(三)】Ubuntu14.04下Caffe利用训练好的模型进行分类的C++接口使用说明(三)
- 使用caffe的Python接口进行训练测试
- 有关Caffe训练好的模型在Python接口下使用分类不准确的问题解决
- 配置caffe python接口时遇到的一些问题
- caffe的python接口学习(9):Caffe在Python中使用内存数据(MemoryData)进行训练
- Mac下配置caffe的Python接口 遇到的问题及解决办法
- Emacs使用anaconda-mode进行Python补全时遇到的问题
- caffe-windows利用python对数据进行可视化
- caffe的python接口学习:用训练好的模型(caffemodel或者h5)来分类新的图片
- Caffe的Python接口进行Cifar10可视化
- 利用Python进行数据分析 中的问题与解决方案汇总
- Caffe在Python中使用内存数据(MemoryData)进行训练
- sqlite3 jdbc、c接口、python接口处理中文时遇到的问题及其解决方法
- Python调用已训练好的caffe模型进行分类
- caffe的python接口学习(6):用训练好的模型(caffemodel)来分类新的图片
- 利用caffe日志进行测试精度训练损失等的画图(caffe训练结果可视化)
- python调用caffe接口进行classify时提示Mean shape incompatible with input shape错误的解决方法