安装MXNet

安装MXNet

前言

    You can run MXNet on Amazon Linux, Ubuntu/Debian, OS X, and Windows operating systems. MXNet can be run on Docker and on Cloud like AWS. MXNet can also be run on embedded devices, such as the Raspberry Pi running Raspbian.
MXNet currently supports the Python, R, Julia and Scala languages.

    你可以在各种操作系统下运行MXNet，诸如Amazon Linux、
Ubuntu/Debian、OS X、和Windows以及云平台AWS，也可以使用Docker等容器打包，还可以运行在如树莓派等嵌入式设备。此外，MXNet有丰富的语言支持，C++、Python、R、Julia和Scala等。

    If you are running Python/R on Amazon Linux or Ubuntu, you can use Git Bash scripts to quickly install the MXNet libraries and all its dependencies.

    如果你使用Python/R语言在Amazon Linux或者Ubuntu上运行，可以使用Git
Bash脚本快速的安装MXNet和相关的依赖库。

    因为个人兴趣，仅仅介绍Debian/Ubuntu系统和Windows系统的安装过程。

 

Prerequisites 基本的环境

Minimum Requirements

    A C++ compiler that supports C++ 11. The C++ compiler compiles and builds MXNet source code. Supported compilers include the following:G++ (4.8 or later),Clang,VS.

    首先要有一个支持C++11特性的编译器，编译和构建MXNet的源码。这样的编译器有：windows下的VS(建议2013以上版本)、两个跨平台的编译器GNU和Clang。

    A BLAS (Basic Linear Algebra Subprograms) library. BLAS libraries contain routines that provide the standard building blocks for performing basic vector and matrix operations. You need a BLAS library to perform basic
linear algebraic operations. Supported BLAS libraries include the following:libblas,openblas,Intel MKL.

    然后，还需要一个BLAS库，就是一个向量和矩阵之间运算的库，如果是windows+intel系统的话，直接按照Intel
MKL的数学核心库，当然也可以使用这些跨软硬件平台的库libblas和openblas，不过如果找不到适合自己系统配置的预编译的文件，需要自己在源码的基础上编译和构建相适应的库。

    最后是两个方便使用MXNet接口的工具：

    Graphviz for visualizing the network graphs.一个图形绘制工具，可以很方便的用来绘制结构化的图形网络。

    Jupyter Notebook for running examples and tutorials of MXNet.

 

Prepare environment for GPU Installation GPU环境的准备

    如果你有GPU可以使用，就准备下CUDA和CUDNN吧，这两个东东是英伟达GPU的开发套件和深度神经网络库，自己到官网找一下适合自己操作系统的版本吧。

    以下是Linux下需要做的相应的配置：

    Unzip the file and change to the cudnn root directory. Move the header and libraries to your local CUDA Toolkit folder:

        tar xvzf cudnn-8.0-linux-x64-v5.1-ga.tgz

        sudo cp -P cuda/include/cudnn.h /usr/local/cuda/include

        sudo cp -P cuda/lib64/libcudnn* /usr/local/cuda/lib64

        sudo chmod a+r /usr/local/cuda/include/cudnn.h

        sudo chmod a+r /usr/local/cuda/lib64/libcudnn*

        sudo ldconfig

    Finally, add configurations to config.mk file:

    If building with GPU support, run below commands to add GPU dependency configurations to config.mk file:

        echo "USE_CUDA=1" >>config.mk

        echo "USE_CUDA_PATH=/usr/local/cuda" >>config.mk

        echo "USE_CUDNN=1" >>config.mk

        cp make/config.mk

    Windows下安装好后环境变量基本也是设计好的。由于本人囊中羞涩，使用的还是十年前奔腾双核的本本，连个独立显卡都没有。

 

Requirements for Computer Vision and Image Augmentation Support

    If you need to support computer vision and image augmentation, you need OpenCV. The Open Source Computer Vision (OpenCV) library contains programming functions for computer vision and image augmentation.

    如果想在计算机视觉和图像处理方面进行扩展的话，需要一个这方面的专业库的支持，OpenCV是一个较好的选择，开源且跨平台。

 

Installing MXNet on Ubuntu在Ubuntu上安装MXNet

    MXNet currently supports Python, R, Julia, and Scala. For users of Python and R on Ubuntu operating systems, MXNet provides a set of Git Bash scripts that installs all of the required MXNet dependencies and the MXNet
library.

    MXNet目前支持Python、R、Julia和Scala等语言，针对在Ubunt上使用Python/R语言的用户，MXNet提供了一个Git
Bash脚本用于快速安装MXNet library以及相关的依赖库。

Install MXNet for Python安装MXNet
（Quick Installation）

    Clone the MXNet source code repository to your computer, run the installation script, and refresh the environment variables. In addition to installing MXNet, the script installs all MXNet dependencies: Numpy, LibBLAS
and OpenCV. It takes around 5 minutes to complete the installation.

    获得源码的途径看自己喜好了，可以安装个git，也可以自己从网站下载。简单的运行安装脚本即可刷新环境变量了，这个脚本可以安装所有MXNet依赖库：Numpy、LibBLAS和OpenCV。话说5分钟即可完成安装(工欲善其事必先利其器)。

        # Install git if not already installed.

        sudo apt-get update

        sudo apt-get -y install git

 

        # Clone mxnet repository. In terminal, run the commands WITHOUT "sudo"

        git clone https://github.com/dmlc/mxnet.git ~/mxnet --recursive

        

        # Install MXNet for Python with all required dependencies

        cd ~/mxnet/setup-utils

        bash install-mxnet-ubuntu-python.sh

     

        # We have added MXNet Python package path in your ~/.bashrc.

        # Run the following command to refresh environment variables.

        $ source ~/.bashrc

    以上这个就是快速的安装，下面分解的分析一下，言称标准安装过程。

 

Standard installation 标准安装

    Installing MXNet is a two-step process:

        Build the shared library from the MXNet C++ source code.

        Install the supported language-specific packages for MXNet.    

    Note: To change the compilation options for your build, edit the make/config.mk file and submit a build request with the make command.

    安装MXNet分为两个部分：使用MXNet C++的源码编译产生可以共享的库libmxnet.so;安装使用MXNet的语言包。

    Build the Shared Library：

    1.获得源码，上面已经介绍了。

    2. you need the following dependencies，安装依赖库，主要是BLAS和OpenCV的库，分别是线性代数子程序库和计算机视觉库。

        sudo apt-get update

        sudo apt-get install -y build-essential git libatlas-base-dev libopencv-dev

    3.将安装的依赖库添加之make文件中以明示：

        cd ~/mxnet

        cp make/config.mk .

        echo "USE_BLAS=openblas" >>config.mk

        echo "ADD_CFLAGS += -I/usr/include/openblas" >>config.mk

        echo "ADD_LDFLAGS += -lopencv_core -lopencv_imgproc -lopencv_imgcodecs" >>config.mk

    4.编译：make -j$(nproc)，执行完以后即可产生库libmxnet.so。

    5.为了方便工作，安装了一些可视化交互的工具：

        sudo apt-get install -y python-pip

        sudo pip install graphviz

        sudo pip install Jupyter

 

    Install the MXNet Package for Python：

    1.安装Python的接口，
Assuming you are in ~/mxnet directory, run below commands.路径依照自己起初放置的。还有一个python语言的科学计算用的库。

    # Install MXNet Python package

    sudo apt-get install python-numpy    

        cd python

        sudo python setup.py install

    2.简单的测试验证下是否安装正确了：If you don’t get an import error, then MXNet is ready for python.如果终端没有提示有错误，那么python版的MXNet已经就绪。

        python

        >>> import mxnet as mx

        >>> a = mx.nd.ones((2, 3), mx.cpu())

        >>> print ((a * 2).asnumpy())

        [[ 2.  2.  2.]

         [ 2.  2.  2.]]

 

Installing MXNet on Windows在Win上安装MXNet

    On Windows, you can download and install the prebuilt MXNet package, or download, build, and install MXNet yourself.

    在Windows，可以直接下载已经编译好的MXNet包，也可以自己编译源码产生与自己系统和开发环境想适应的库。

 

Installing the Prebuilt Package on Windows

    MXNet provides a prebuilt package for Windows. The prebuilt package includes the MXNet library, all of the dependent third-party libraries, a sample C++ solution for Visual Studio, and the Python installation script.
To install the prebuilt package:

    1.Download the latest prebuilt package from the Releases tab of MXNet. There are two versions. One with GPU support (using CUDA and CUDNN v3), and one without GPU support. Choose the version that suits your hardware
configuration. For more information on which version works on each hardware configuration, see Requirements for GPU.

    2.Unpack the package into a folder, with an appropriate name, such as D:\MXNet.

    3.Open the folder, and install the package by double-clicking setupenv.cmd. This sets up all of the environment variables required by MXNet.

    4.Test the installation by opening the provided sample C++ Visual Studio solution and building it.

    This produces a library called libmxnet.dll.

    MXNet提供了基于Windows的预先编译好的包，其中有MXNet的库以及所需的第三方的库。还包含了一个VS的C++工程和python的安装脚本。

    1.可以从官网下载最新的MXNet包，有两个版本，一个gpu，一个不支持。

    2.3.将包解压之相应的文件夹下面，使用其中的setupenv.cmd设置MXNet所需的所有的环境变量。

       4.通过包里面包含的VS工程进行test。

 

Building and Installing Packages on Windows

    To build and install MXNet yourself, you need the following dependencies. Install the required dependencies:

    为了在自己的系统上构建MXNet，需要安装下面的依赖库：

    1.If Microsoft Visual Studio 2013 is not already installed, download and install it. You can download and install the free community edition.VS2013,这个应该都会有了。

    2.Install Visual C++ Compiler Nov 2013 CTP.这个是在C++编译器上有需要的新特性。后面的3项和4项便是替换掉VS2013原来的C++编译器。

    3.Back up all of the files in the C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC folder to a different location.

    4.Copy all of the files in the C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP folder (or the folder where you extracted the zip archive) to the C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC
folder, and overwrite all existing files.

    5.Download and install OpenCV.

    6.Unzip the OpenCV package.

    7.Set the environment variable OpenCV_DIR to point to the OpenCV build directory.OpenCV官方有现成的安装包，但有时候并不一定适合你的这个开发环境，建议下载源码在自己的开发环境下重新构建。

    8.If you don’t have the Intel Math Kernel Library (MKL) installed, download and install OpenBlas.至于Blas库，有很多实现的版本，不同的版本有不同的性能，MKL是intel针对自己的处理器开发的，性能还可以。

    9.Set the environment variable OpenBLAS_HOME to point to the OpenBLAS directory that contains the include and lib directories. Typically, you can find the directory in C:\Program files (x86)\OpenBLAS\.设置Blas库的环境变量。

    10.Download and install CuDNN. To get access to the download link, register as an NVIDIA community user.如果使用GPU的话，这个是英伟达有关深度神经网络的库，有关其Blas和fft的库是直接包含在开发套件cuda中的，前面已有介绍具体的安装。

    After you have installed all of the required dependencies, build the MXNet source code:

    在以上所需的依赖库安装好后，下面就是MXNet
源码的构建了：

    1.Download the MXNet source code from GitHub.获取源码。

    2.Use CMake to create a Visual Studio solution in ./build.使用CMake构建，这是windows下的make方式。记得选择构建的编译器是刚刚安装的VS2013.

    3.In Visual Studio, open the solution file,.sln, and compile it. These commands produce a library called mxnet.dll in the ./build/Release/ or ./build/Debug folder.在CMake构建好后，直接找寻VS2013的工程，编译产生相应的mxnet.dll。

    Next, we install graphviz library that we use for visualizing network graphs you build on MXNet. We will also install Jupyter Notebook used for running MXNet tutorials and examples.

    下面，将安装graphviz和Jupyter。

    1.Install graphviz by downloading MSI installer from Graphviz Download Page. Note Make sure to add graphviz executable path to PATH environment variable.直接找寻msi的安装包，记得设置环境变量。

    2.Install Jupyter by installing Anaconda for Python 2.7 Note Do not install Anaconda for Python 3.5. MXNet has few compatibility issue with Python 3.5.通过安装Anaconda的windows包即可获取Jupyter，记得是支持Python
2.7的安装包。

    We have installed MXNet core library. Next, we will install MXNet interface package for programming language of your choice:python。

    1.Install Python using windows installer available.Python 2.7的安装

    2.Install Numpy using windows installer available.通过安装Anaconda的windows包即可获取Numpy
。

    3.Next, we install Python package interface for MXNet. You can find the Python interface package for MXNet on GitHub.在MXNet
的源码中设置支持Python的接口。

        # Assuming you are in root mxnet source code folder

        cd python

        sudo python setup.py install

    Done! We have installed MXNet with Python interface. Run below commands to verify our installation is successful.

        # Open Python terminal

        python

    核验是否安装成功了：

        # You should be able to import mxnet library without anyissues.

        >>> import mxnet as mx;

        >>> a = mx.nd.ones((2, 3));

        >>> print ((a*2).asnumpy());

            [[ 2.  2.  2.]

            [ 2.  2.  2.]]

 

总结

    上面所需的资源，如果能够找寻到现成适合自己系统配置的预编译好的包，安装过程会省事许多。但是我还是建议自己编译和构建整个系统，因为这是进入大牛的必备技能。

    CMake是个比较方便的工具，对命令行不熟悉的可以使用其图像界面，这个工具对windows和Linux都是很好使的。