Apache Mxnet 简明教程

Apache MXNet - Introduction

本章重点介绍了 Apache MXNet 的功能，并讨论了该深度学习软件框架的最新版本。

This chapter highlights the features of Apache MXNet and talks about the latest version of this deep learning software framework.

What is MXNet?

Apache MXNet 是一款功能强大的开源深度学习软件框架工具，旨在帮助开发人员构建、训练和部署深度学习模型。在过去的几年里，从医疗保健到交通再到制造业，事实上，在我们的日常生活各个方面，深度学习的影响都已得到广泛普及。如今，深度学习被公司用于解决一些难题，如人脸识别、物体检测、光学字符识别 (OCR)、语音识别和机器翻译。

Apache MXNet is a powerful open-source deep learning software framework instrument helping developers build, train, and deploy Deep Learning models. Past few years, from healthcare to transportation to manufacturing and, in fact, in every aspect of our daily life, the impact of deep learning has been widespread. Nowadays, deep learning is sought by companies to solve some hard problems like Face recognition, object detection, Optical Character Recognition (OCR), Speech Recognition, and Machine Translation.

这就是 Apache MXNet 受支持的原因：

That’s the reason Apache MXNet is supported by:

  1. Some big companies like Intel, Baidu, Microsoft, Wolfram Research, etc.

  2. Public cloud providers including Amazon Web Services (AWS), and Microsoft Azure

  3. Some big research institutes like Carnegie Mellon, MIT, the University of Washington, and the Hong Kong University of Science & Technology.

Why Apache MXNet?

当时，已经存在各种深度学习平台，如 Torch7、Caffe、Theano、TensorFlow、Keras、Microsoft Cognitive Toolkit 等，您可能想知道为什么选择 Apache MXNet？我们来了解一下背后的原因：

There are various deep learning platforms like Torch7, Caffe, Theano, TensorFlow, Keras, Microsoft Cognitive Toolkit, etc. existed then you might wonder why Apache MXNet? Let’s check out some of the reasons behind it:

  1. Apache MXNet solves one of the biggest issues of existing deep learning platforms. The issue is that in order to use deep learning platforms one must need to learn another system for a different programming flavor.

  2. With the help of Apache MXNet developers can exploit the full capabilities of GPUs as well as cloud computing.

  3. Apache MXNet can accelerate any numerical computation and places a special emphasis on speeding up the development and deployment of large-scale DNN (deep neural networks).

  4. It provides the users the capabilities of both imperative and symbolic programming.

Various Features

如果您正在寻找一个灵活的深度学习库来快速开发前沿的深度学习研究，或者一个强大的平台来推动生产工作负载，那么 Apache MXNet 就是您搜索之旅的终点。这是因为它具有以下特点：

If you are looking for a flexible deep learning library to quickly develop cutting-edge deep learning research or a robust platform to push production workload, your search ends at Apache MXNet. It is because of the following features of it:

Distributed Training

无论是在具有近似线性缩放效率的多 GPU 或多主机训练中，Apache MXNet 都允许开发人员最大程度地利用其硬件。MXNet 也支持与 Horovod 集成，Horovod 是优步创建的一个开源分布式深度学习框架。

Whether it is multi-gpu or multi-host training with near-linear scaling efficiency, Apache MXNet allows developers to make most out of their hardware. MXNet also support integration with Horovod, which is an open source distributed deep learning framework created at Uber.

针对此次集成，以下是一些 Horovod 中定义的常见分布式 API：

For this integration, following are some of the common distributed APIs defined in Horovod:

  1. horovod.broadcast()

  2. horovod.allgather()

  3. horovod.allgather()

在这方面，MXNet 为我们提供了以下功能：

In this regard, MXNet offer us the following capabilities:

  1. Device Placement − With the help of MXNet we can easily specify each data structure (DS).

  2. Automatic Differentiation − Apache MXNet automates the differentiation i.e. derivative calculations.

  3. Multi-GPU training − MXNet allows us to achieve scaling efficiency with number of available GPUs.

  4. Optimized Predefined Layers − We can code our own layers in MXNet as well as the optimized the predefined layers for speed also.

Hybridization

Apache MXNet 为其用户提供了混合前端。在 Gluon Python API 的帮助下，它可以弥合理命令和符号功能之间的差距。可以通过调用其混合功能来完成。

Apache MXNet provides its users a hybrid front-end. With the help of the Gluon Python API it can bridge the gap between its imperative and symbolic capabilities. It can be done by calling it’s hybridize functionality.

Faster Computation

线性运算（如几十个或几百个矩阵乘法）是深度神经网络的计算瓶颈。为了解决这个瓶颈问题，MXNet 提供了：

The linear operations like tens or hundreds of matrix multiplications are the computational bottleneck for deep neural nets. To solve this bottleneck MXNet provides −

  1. Optimized numerical computation for GPUs

  2. Optimized numerical computation for distributed ecosystems

  3. Automation of common workflows with the help of which the standard NN can be expressed briefly.

Language Bindings

MXNet 与 Python 和 R 等高级语言深度集成。它还为其他编程语言（如以下语言）提供支持：

MXNet has deep integration into high-level languages like Python and R. It also provides support for other programming languages such as-

  1. Scala

  2. Julia

  3. Clojure

  4. Java

  5. C/C++

  6. Perl

我们无需学习任何新的编程语言，而且 MXNet 与混合特性相结合，可以非常流畅地从 Python 过渡到我们选择的编程语言的部署。

We do not need to learn any new programming language instead MXNet, combined with hybridization feature, allows an exceptionally smooth transition from Python to deployment in the programming language of our choice.

Latest version MXNet 1.6.0

Apache Software Foundation (ASF) 已于 2020 年 2 月 21 日根据 Apache License 2.0 发布了 Apache MXNet 的稳定版本 1.6.0。这是支持 Python 2 的最后一个 MXNet 版本，因为 MXNet 社区投票决定在以后的版本中不再支持 Python 2。让我们了解一下此版本为其用户带来的一些新特性。

Apache Software Foundation (ASF) has released the stable version 1.6.0 of Apache MXNet on 21st February 2020 under Apache License 2.0. This is the last MXNet release to support Python 2 as MXNet community voted to no longer support Python 2 in further releases. Let us check out some of the new features this release brings for its users.

NumPy-Compatible interface

NumPy 由于其灵活性与普遍性，已得到机器学习从业者、科学家和学生们的广泛使用。但我们知道，如今像图形处理单元 (GPU) 这样的硬件加速器已逐渐集成到各种机器学习 (ML) 工具包中，为了利用 GPU 的速度，NumPy 用户需要切换到具有不同语法的框架。

Due to its flexibility and generality, NumPy has been widely used by Machine Learning practitioners, scientists, and students. But as we know that, these days’ hardware accelerators like Graphical Processing Units (GPUs) have become increasingly assimilated into various Machine Learning (ML) toolkits, the NumPy users, to take advantage of the speed of GPUs, need to switch to new frameworks with different syntax.

凭借 MXNet 1.6.0，Apache MXNet 正朝着一个 NumPy 兼容的编程体验迈进。新界面为熟悉 NumPy 语法的从业者提供了同等的可使用性和表现力。除此之外，MXNet 1.6.0 还让现有的 Numpy 系统能够利用 GPU 等硬件加速器来加速大规模计算。

With MXNet 1.6.0, Apache MXNet is moving toward a NumPy-compatible programming experience. The new interface provides equivalent usability as well as expressiveness to the practitioners familiar with NumPy syntax. Along with that MXNet 1.6.0 also enables the existing Numpy system to utilize hardware accelerators like GPUs to speed-up large-scale computations.

Integration with Apache TVM

Apache TVM 是一个针对 CPU、GPU 和专用加速器等硬件后端的开源端到端深度学习编译器堆栈，其目标是填补以生产力为重点的深度学习框架和面向性能的硬件后端之间的空白。借助最新版本 MXNet 1.6.0，用户可以利用 Apache(incubating) TVM 在 Python 编程语言中实现高性能操作内核。这项新功能的两个主要优点包括：

Apache TVM, an open-source end-to-end deep learning compiler stack for hardware-backends such as CPUs, GPUs, and specialized accelerators, aims to fill the gap between the productivity-focused deep-learning frameworks and performance-oriented hardware backends. With the latest release MXNet 1.6.0, users can leverage Apache(incubating) TVM to implement high-performance operator kernels in Python programming language. Two main advantages of this new feature are following −

  1. Simplifies the former C++ based development process.

  2. Enables sharing the same implementation across multiple hardware backend such as CPUs, GPUs, etc.

Improvements on existing features

除了上述 MXNet 1.6.0 的特性外，它还对现有特性进行了一些改进。改进内容如下：

Apart from the above listed features of MXNet 1.6.0, it also provides some improvements over the existing features. The improvements are as follows −

Grouping element-wise operation for GPU

正如我们所知，逐元素运算的性能取决于内存带宽，因此，链接此类运算可能会降低整体性能。Apache MXNet 1.6.0 实现了逐元素运算融合，这实际上会按需生成即时融合运算。此类逐元素运算融合还能减少存储需求并提高整体性能。

As we know the performance of element-wise operations is memory-bandwidth and that is the reason, chaining such operations may reduce overall performance. Apache MXNet 1.6.0 does element-wise operation fusion, that actually generates just-in-time fused operations as and when possible. Such element-wise operation fusion also reduces storage needs and improve overall performance.

Simplifying common expressions

MXNet 1.6.0 消除冗余表达式，并简化通用表达式。这种增强还有利于提高内存使用情况和全部执行时间。

MXNet 1.6.0 eliminates the redundant expressions and simplify the common expressions. Such enhancement also improves memory usage and total execution time.

Optimizations

MXNet 1.6.0 还对现有特性和操作符提供多种优化，如下所示：

MXNet 1.6.0 also provides various optimizations to existing features & operators, which are as follows:

  1. Automatic Mixed Precision

  2. Gluon Fit API

  3. MKL-DNN

  4. Large tensor Support

  5. TensorRT integration

  6. Higher-order gradient support

  7. Operators

  8. Operator performance profiler

  9. ONNX import/export

  10. Improvements to Gluon APIs

  11. Improvements to Symbol APIs

  12. More than 100 bug fixes