caffe的搭建过程以及遇上的各种问题的汇总
2016-09-21 14:47
190 查看
Caffe的搭建过程以及遇上的各种问题的汇总:
ubuntu15.04+CUDA7.5+opencv3.0.0+python2.7
整个搭建过程参考 http://blog.csdn.net/ubunfans/article/details/47724341,过程中遇上了各种花式的问题,废了两天终于将caffe框架搭好。整理一下过程和遇到的问题,给自己长长经验。
caffe安装步骤:
(声明:从第一步到第六步跟我上面提及到的那个博客是一样的,安装过程可以自行参考。经验:除非框架有特殊要求,否则其他例如cuda,opencv,python等尽量不要装最新的版本,毕竟刚刚出来的版本后续的都没有跟上,会很大程度的加大出错率和搭建的苦难度。)
1.安装开发所需的依赖项:
2.安装CUDA7.5:
安装cuda也很简单,到cuda官网 下载你需要的版本,然后按照它提示的那几句代码安装就可以啦,建议下deb包离线安装。CUDA的deb包比较大,用ubuntu系统下载的话速度慢,建议用windows下载再拷过去(后面遇上比较大的文件下载我都是用windows下的)。。
官网和我参考的博客都提及到用md5检验下下载的deb包,这个自行选择啦~
下载到deb包后,cd到所在的目录,执行下面的语句:
装好后最好重启一下系统再接着接下来的步骤。
3.安装cudnn
到nvidia官网下载cudnn(先注册,后下载。我下载的是cudnn-7.5-linux-x64-v5.1.tgz)
4.设置环境变量
添加cuda环境变量:
sudo gedit /etc/profile
添加内容:
PATH=/usr/local/cuda/bin:$PATH
export PATH
使环境变量生效:
source /etc/profile
添加lib库路径:
sudo gedit /etc/ld.so.conf.d/cuda.conf
添加内容:
/usr/local/cuda/lib64
使路径生效:
sudo ldconfig
5.安装cuda samples
如果前面的都安装成功,此处可以显示显卡的信息:
6.安装atlas
安装命令:
sudo apt-get install libatlas-base-dev
7.安装opencv
(此处开始和参考博客有不同,之前按照原来博客的方法安装在编译的时候出了一些错误,也可能是版本问题)
我安装的版本是opencv3.0.0, 在opencv官网 下载,解压。
装好opencv的各种依赖项
(可以把代码复制下来,在opencv的文件夹里面创建一个dependencies.sh, 然后执行sudo bash dependencies.sh)
装好依赖项后,执行下面的语句安装:
cmake 里面的不同点是加入了-D BUILD_TIFF=ON ,这个如果没有加入的话在caffe的编译阶段会针对Opencv报错。
安装好Opencv可以找一个网上的例子测试一下是否安装成功。
8.安装python
到anaconda官网 下载,我装的是python2.7
切换到下载文件目录,执行:
sudo bash Anaconda2-4.1.1-Linux-x86_64.sh
添加路径:
执行sudo gedit /etc/ld.so.conf
添加内容:
/home/username/anaconda/lib 注意改username
执行sudo gedit ~/.bashrc
添加内容:
export LD_LIBRARY_PATH="/home/username/anaconda/lib:$LD_LIBRARY_PATH"
9.安装python依赖库
先下载个caffe包 github-caffe ,解压后进入caffe-master/python,执行:
for req in $(cat requirements.txt); do pip install $req; done
10.修改Makefile和Mafile.config
首先进入caffe-master目录,复制一份Makefile.config.examples
cp Makefile.config.example Makefile.config
改动1:打开Makefile,将第181行的:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hlhdf5
改为:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs
这里的改动是为了避免出现hdf5.h错误。
改动2:打开Makefile.config,将第90行的:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
改为:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
改动理由同上。
Makefile.config里面的设置也要根据自己的配置和装的版本,选择每一项是否去掉# 或加上#。
例如我用的是ubuntu15.04, python2.7, cuda7.5,opencv 3.0.0,因此我的Makefile.config为:
里面的内容一定要检查清楚,尤其检查phthon的目录,lib的路径有没有错误,这些有可能导致接下来编译可以通过但是安装python接口的时候出现找不到python.h的错误等等。
11.编译caffe
所有都整理完成之后就进入编译:
如果编译中出现什么错误,需要重新编译,一定要先make clean。
91c4
12.编译python wrapper
虽然这里命令很简单:
sudo make pycaffe
但是编译完成后,进入ipython, 输入import caffe的时候会发现出现了
ImportError: No module
named google.protobuf
这是因为protobuf版本过低的原因,需要手动装一下protobuf:
先到github-protobuf 下载,解压。
执行:
发现./autogen.sh的时候可能会有错误,这是因为curl没装好,可以选择修复,也可以选择手动下载autogen.sh里面提到的两个包,手动解压,然后将他们按照autogen.sh里面要求重命名和移动。将gmock-release的那个包重命名为gmock,将gtest-release的包重命名为gtest,放到gmock里面。再执行下面的命令
编译完成后,去到python的目录,执行
sudo python setup.py install
这样,整个接口就完成了,再重新打开ipython, import caffe就不会再报错。
至此,整个caffe的安装配置工作完成!
ubuntu15.04+CUDA7.5+opencv3.0.0+python2.7
整个搭建过程参考 http://blog.csdn.net/ubunfans/article/details/47724341,过程中遇上了各种花式的问题,废了两天终于将caffe框架搭好。整理一下过程和遇到的问题,给自己长长经验。
caffe安装步骤:
(声明:从第一步到第六步跟我上面提及到的那个博客是一样的,安装过程可以自行参考。经验:除非框架有特殊要求,否则其他例如cuda,opencv,python等尽量不要装最新的版本,毕竟刚刚出来的版本后续的都没有跟上,会很大程度的加大出错率和搭建的苦难度。)
1.安装开发所需的依赖项:
sudo apt-get install build-essential # basic requirement sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler #required by caffe
2.安装CUDA7.5:
安装cuda也很简单,到cuda官网 下载你需要的版本,然后按照它提示的那几句代码安装就可以啦,建议下deb包离线安装。CUDA的deb包比较大,用ubuntu系统下载的话速度慢,建议用windows下载再拷过去(后面遇上比较大的文件下载我都是用windows下的)。。
官网和我参考的博客都提及到用md5检验下下载的deb包,这个自行选择啦~
下载到deb包后,cd到所在的目录,执行下面的语句:
sudo dpkg -i cuda-repo-__.deb sudo apt-get update sudo apt-get install cuda
装好后最好重启一下系统再接着接下来的步骤。
3.安装cudnn
到nvidia官网下载cudnn(先注册,后下载。我下载的是cudnn-7.5-linux-x64-v5.1.tgz)
tar -zxvf cudnn-7.5-linux-x64-v5.1.tgz cd cuda sudo cp lib64/lib* /usr/local/cuda/lib64/ sudo cp include/cudnn.h /usr/local/cuda/include/ #更新软链接 cd /usr/local/cuda/lib64/ sudo chmod +r libcudnn.so.5.1.3 #注意版本号,不一样记得要改 sudo ln -sf libcudnn.so.5.1.3 libcudnn.so.5 sudo ln -sf libcudnn.so.5 libcudnn.so sudo ldconfig
4.设置环境变量
添加cuda环境变量:
sudo gedit /etc/profile
添加内容:
PATH=/usr/local/cuda/bin:$PATH
export PATH
使环境变量生效:
source /etc/profile
添加lib库路径:
sudo gedit /etc/ld.so.conf.d/cuda.conf
添加内容:
/usr/local/cuda/lib64
使路径生效:
sudo ldconfig
5.安装cuda samples
cd /usr/local/cuda/samples #bulid samples sudo make all -j4 #编译完成后 cd bin/x86_64/linux/release ./deviceQuery
如果前面的都安装成功,此处可以显示显卡的信息:
./deviceQuery Starting... CUDA Device Query (Runtime API) version (CUDART static linking) Detected 1 CUDA Capable device(s) Device 0: "GeForce GTX 980" CUDA Driver Version / Runtime Version 7.5 / 7.5 CUDA Capability Major/Minor version number: 5.2 Total amount of global memory: 4093 MBytes (4291493888 bytes) (16) Multiprocessors, (128) CUDA Cores/MP: 2048 CUDA Cores GPU Max Clock rate: 1216 MHz (1.22 GHz) Memory Clock rate: 3505 Mhz Memory Bus Width: 256-bit L2 Cache Size: 2097152 bytes Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096) Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 163 4000 84), 2048 layers Total amount of constant memory: 65536 bytes Total amount of shared memory per block: 49152 bytes Total number of registers available per block: 65536 Warp size: 32 Maximum number of threads per multiprocessor: 2048 Maximum number of threads per block: 1024 Max dimension size of a thread block (x,y,z): (1024, 1024, 64) Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535) Maximum memory pitch: 2147483647 bytes Texture alignment: 512 bytes Concurrent copy and kernel execution: Yes with 2 copy engine(s) Run time limit on kernels: Yes Integrated GPU sharing Host Memory: No Support host page-locked memory mapping: Yes Alignment requirement for Surfaces: Yes Device has ECC support: Disabled Device supports Unified Addressing (UVA): Yes Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0 Compute Mode: < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) > deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 7.5, CUDA Runtime Version = 7.5, NumDevs = 1, Device0 = GeForce GTX 980 Result = PASS
6.安装atlas
安装命令:
sudo apt-get install libatlas-base-dev
7.安装opencv
(此处开始和参考博客有不同,之前按照原来博客的方法安装在编译的时候出了一些错误,也可能是版本问题)
我安装的版本是opencv3.0.0, 在opencv官网 下载,解压。
装好opencv的各种依赖项
echo "--- Removing any pre-installed ffmpeg and x264" sudo apt-get -qq remove ffmpeg x264 libx264-dev echo "--- Installing dependency: " sudo apt-get -y install libopencv-dev sudo apt-get -y install build-essential sudo apt-get -y install checkinstall sudo apt-get -y install cmake sudo apt-get -y install pkg-config sudo apt-get -y install yasm sudo apt-get -y install libtiff4-dev sudo apt-get -y install libjpeg-dev sudo apt-get -y install libjasper-dev sudo apt-get -y install libavcodec-dev sudo apt-get -y install libavformat-dev sudo apt-get -y install libswscale-dev sudo apt-get -y install libdc1394-22-dev sudo apt-get -y install libxine-dev sudo apt-get -y install libgstreamer0.10-dev sudo apt-get -y install libgstreamer-plugins-base0.10-dev sudo apt-get -y install libv4l-dev sudo apt-get -y install python-dev sudo apt-get -y install python-numpy sudo apt-get -y install libtbb-dev sudo apt-get -y install libqt4-dev sudo apt-get -y install libgtk2.0-dev sudo apt-get -y install libfaac-dev sudo apt-get -y install libmp3lame-dev sudo apt-get -y install libopencore-amrnb-dev sudo apt-get -y install libopencore-amrwb-dev sudo apt-get -y install libtheora-dev sudo apt-get -y install libvorbis-dev sudo apt-get -y install libxvidcore-dev sudo apt-get -y install x264 sudo apt-get -y install v4l-utils sudo apt-get -y install ffmpeg sudo apt-get -y install unzip
(可以把代码复制下来,在opencv的文件夹里面创建一个dependencies.sh, 然后执行sudo bash dependencies.sh)
装好依赖项后,执行下面的语句安装:
mkdir build cd build cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_OPENGL=ON -D WITH_CUFFT=ON -D WITH_CUDA=ON -D WITH_FFMPEG=OFF -D BUILD_TIFF=ON .. make -j4 sudo make install sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf' sudo ldconfig echo "OpenCV 3.0.0-rc1 ready to be used"
cmake 里面的不同点是加入了-D BUILD_TIFF=ON ,这个如果没有加入的话在caffe的编译阶段会针对Opencv报错。
安装好Opencv可以找一个网上的例子测试一下是否安装成功。
8.安装python
到anaconda官网 下载,我装的是python2.7
切换到下载文件目录,执行:
sudo bash Anaconda2-4.1.1-Linux-x86_64.sh
添加路径:
执行sudo gedit /etc/ld.so.conf
添加内容:
/home/username/anaconda/lib 注意改username
执行sudo gedit ~/.bashrc
添加内容:
export LD_LIBRARY_PATH="/home/username/anaconda/lib:$LD_LIBRARY_PATH"
9.安装python依赖库
先下载个caffe包 github-caffe ,解压后进入caffe-master/python,执行:
for req in $(cat requirements.txt); do pip install $req; done
10.修改Makefile和Mafile.config
首先进入caffe-master目录,复制一份Makefile.config.examples
cp Makefile.config.example Makefile.config
改动1:打开Makefile,将第181行的:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hlhdf5
改为:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs
这里的改动是为了避免出现hdf5.h错误。
改动2:打开Makefile.config,将第90行的:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
改为:
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
改动理由同上。
Makefile.config里面的设置也要根据自己的配置和装的版本,选择每一项是否去掉# 或加上#。
例如我用的是ubuntu15.04, python2.7, cuda7.5,opencv 3.0.0,因此我的Makefile.config为:
## Refer to http://caffe.berkeleyvision.org/installation.html # Contributions simplifying and improving our build system are welcome! # cuDNN acceleration switch (uncomment to build with cuDNN). USE_CUDNN := 1 # CPU-only switch (uncomment to build without GPU support). # CPU_ONLY := 1 # uncomment to disable IO dependencies and corresponding data layers # USE_OPENCV := 0 # USE_LEVELDB := 0 # USE_LMDB := 0 # uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary) # You should not set this flag if you will be reading LMDBs with any # possibility of simultaneous read and write # ALLOW_LMDB_NOLOCK := 1 # Uncomment if you're using OpenCV 3 OPENCV_VERSION := 3 # To customize your choice of compiler, uncomment and set the following. # N.B. the default for Linux is g++ and the default for OSX is clang++ # CUSTOM_CXX := g++ # CUDA directory contains bin/ and lib/ directories that we need. CUDA_DIR := /usr/local/cuda # On Ubuntu 14.04, if cuda tools are installed via # "sudo apt-get install nvidia-cuda-toolkit" then use this instead: # CUDA_DIR := /usr # CUDA architecture setting: going with all of them. # For CUDA < 6.0, comment the *_50 lines for compatibility. CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \ -gencode arch=compute_20,code=sm_21 \ -gencode arch=compute_30,code=sm_30 \ -gencode arch=compute_35,code=sm_35 \ -gencode arch=compute_50,code=sm_50 \ -gencode arch=compute_50,code=compute_50 # BLAS choice: # atlas for ATLAS (default) # mkl for MKL # open for OpenBlas BLAS := atlas # Custom (MKL/ATLAS/OpenBLAS) include and lib directories. # Leave commented to accept the defaults for your choice of BLAS # (which should work)! # BLAS_INCLUDE := /path/to/your/blas # BLAS_LIB := /path/to/your/blas # Homebrew puts openblas in a directory that is not on the standard search path # BLAS_INCLUDE := $(shell brew --prefix openblas)/include # BLAS_LIB := $(shell brew --prefix openblas)/lib # This is required only if you will compile the matlab interface. # MATLAB directory should contain the mex binary in /bin. # MATLAB_DIR := /usr/local # MATLAB_DIR := /Applications/MATLAB_R2012b.app # NOTE: this is required only if you will compile the python interface. # We need to be able to find Python.h and numpy/arrayobject.h. #PYTHON_INCLUDE := /usr/include/python2.7 \ /usr/lib/python2.7/dist-packages/numpy/core/include # Anaconda Python distribution is quite popular. Include path: # Verify anaconda location, sometimes it's in root. ANACONDA_HOME := $(HOME)/anaconda PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ $(ANACONDA_HOME)/include/python2.7 \ $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \ # Uncomment to use Python 3 (default is Python 2) # PYTHON_LIBRARIES := boost_python3 python3.5m # PYTHON_INCLUDE := $(ANACONDA_HOME)/include/python3.5m \ # $(ANACONDA_HOME)/lib/python3.5/dist-packages/numpy/core/include # We need to be able to find libpythonX.X.so or .dylib. #PYTHON_LIB := /usr/lib PYTHON_LIB := $(ANACONDA_HOME)/lib # Homebrew installs numpy in a non standard path (keg only) # PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include # PYTHON_LIB += $(shell brew --prefix numpy)/lib # Uncomment to support layers written in Python (will link against Python libs) # WITH_PYTHON_LAYER := 1 # Whatever else you find you need goes here. INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/ LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib # If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies # INCLUDE_DIRS += $(shell brew --prefix)/include # LIBRARY_DIRS += $(shell brew --prefix)/lib # Uncomment to use `pkg-config` to specify OpenCV library paths. # (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.) # USE_PKG_CONFIG := 1 # N.B. both build and distribute dirs are cleared on `make clean` BUILD_DIR := build DISTRIBUTE_DIR := distribute # Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171 # DEBUG := 1 # The ID of the GPU that 'make runtest' will use to run unit tests. TEST_GPUID := 0 # enable pretty build (comment to see full commands) Q ?= @
里面的内容一定要检查清楚,尤其检查phthon的目录,lib的路径有没有错误,这些有可能导致接下来编译可以通过但是安装python接口的时候出现找不到python.h的错误等等。
11.编译caffe
所有都整理完成之后就进入编译:
make all -j4 make test make runtest
如果编译中出现什么错误,需要重新编译,一定要先make clean。
91c4
12.编译python wrapper
虽然这里命令很简单:
sudo make pycaffe
但是编译完成后,进入ipython, 输入import caffe的时候会发现出现了
ImportError: No module
named google.protobuf
这是因为protobuf版本过低的原因,需要手动装一下protobuf:
先到github-protobuf 下载,解压。
执行:
./autogen.sh ./configure make make check sudo make install
发现./autogen.sh的时候可能会有错误,这是因为curl没装好,可以选择修复,也可以选择手动下载autogen.sh里面提到的两个包,手动解压,然后将他们按照autogen.sh里面要求重命名和移动。将gmock-release的那个包重命名为gmock,将gtest-release的包重命名为gtest,放到gmock里面。再执行下面的命令
编译完成后,去到python的目录,执行
sudo python setup.py install
这样,整个接口就完成了,再重新打开ipython, import caffe就不会再报错。
至此,整个caffe的安装配置工作完成!
相关文章推荐
- ubuntu16.04安装caffe以及各种问题汇总
- ubuntu16.04安装caffe以及各种问题汇总
- cocos2d-x-2.2.0_win7+vs2010搭建_eclipse+ndk-r9+cygwin搭建_教程以及编译问题汇总
- qt4嵌入式开发环境搭建过程(包括各种异常问题)(2)
- Windows下搭建Git服务器各种问题汇总(一)
- 搭建简单FTP服务器以及过程中容易遇到的几个问题(一)
- Ceph搭建过程中遇到的各种问题
- cocos2d-x-2.2.0_win7+vs2010搭建_eclipse+ndk-r9+cygwin搭建_教程以及编译问题汇总
- play框架配置使用过程中遇到的各种疑难问题汇总(学习同事经验)
- SQL Server数据库启动过程,以及启动不起来的各种问题的分析及解决技巧
- caffe安装过程中碰到的一些问题以及解决方案
- caffe安装过程中遇到的问题以及解决方法
- cocos2d-x-2.2.0_win7+vs2010搭建_eclipse+ndk-r9+cygwin搭建_教程以及编译问题汇总
- 你所不知道的SQL Server数据库启动过程,以及启动不起来的各种问题的分析及解决技巧
- 【Android】Intent 页面跳转问题以及 会跳刷新 等各种问题汇总
- qt4嵌入式开发环境搭建过程(包括各种异常问题)(1)
- 你所不知道的SQL Server数据库启动过程,以及启动不起来的各种问题的分析及解决技巧
- 复制虚拟机vmware centos搭建集群节点过程中网络配置eth0和eth1遇到的问题以及NAT模式下虚拟机静态IP配置方法
- Cocoa pods的安装使用以及过程中可能会遇到问题的汇总
- Redis 3.0.2集群搭建以及相关问题汇总