1How deep learning works — The geometry of deep learningXiao Dong, Jiasong Wu, Ling ZhouFaculty of Computer Science and Engineering, Southeast University, Nanjing, ChinaWhy and how that deep learning works well on different tasks remains a mystery from a theoretical perspective. In this paperwe draw a geometric picture of the deep learning system by f i nding its analogies with two existing geometric structures, thegeometry of quantum computations and the geometry of the diffeomorphic template matching. In this framework, we give thegeometric structures of different deep learning systems including convolutional neural networks, residual networks, recursive neuralnetworks, recurrent neural networks and the equilibrium prapagation framework. We can also analysis the relationship betweenthe geometrical structures and their performance of different networks in an algorithmic level so that the geometric frameworkmay guide the design of the structures and algorithms of deep learning systems.Index Terms—Deep learning, geometry, quantum computation, computational anatomyI. I NTRODUCTIONIn the last decade, deep learning systems show a fascinatingperformance in solving different complex tasks. We havedesigned different system structures for different problems andrevealed some general rules for the designing of deep learningsystems. There are also theoretical attempts to understanddeep learning systems from both mathematical and physicalperspectives[1]. But still we are lacking of a theoreticalframework to answer the question, why and how deep learningsystems works. Also it’s highly desired that a theoreticalframework of deep learning systems can be used to guidethe design the structures of deep learning systems from analgorithmic level.In this paper we try to f ill this gap by drawing a geometricpicture of deep learning systems. We build our geometricframework to understand deep learning systems by comparingthe deep learning system with other two existing maturalgeometric structures, the geometry of quantum computationsand the geometry of diffeomorphic template matching in thef ield of computational anatomy. We show that deep learningsystems can be formulated in a geometric language, by whichwe can draw geometric pictures of different deep learningsystems including convolutional neural networks, residualnetworks, recursive neural networks, fractal neural networksand recurrent neural networks. What’s more, these geometricpictures can be used to analysis the performance of differentdeep learning structures and provide guidance to the design ofdeep learning systems from an algorithmic level.The rest of this paper is arranged as follows. We willf irst give a brief overview of the geometry of quantumcomputations and the geometry of diffeomorphic templatematching. Then we will explain the geometric frameworkof deep learning systems and apply our framework to drawcorrespondent geometric pictures of different deep learningnetworks. Finally we will give a general optimization basedframework of deep learning systems, which can be used toaddress the equilibrium propagation algorithm.II. W HAT CAN GEOMETRY TEACH USIt’s well known that geometry is not only a core concept ofmathematics, but also it plays a key role in modern physics.The great success of geometrization of physics tells us thatthe soul of physical systems lies in their geometric structures.It’s natural to ask if geometry can help to reveal the secretof our human intelligence and our state-of-the-art artif icialintelligence systems, deep learning systems. The answer isYES.We will f irst introduce two interesting geometric structuresin the f ields of quantum computation and computationalanatomy. We will see, their geometric structures share somesimilarities with deep learning systems and the geometricframework of deep learning systems can be built based onan understanding of these structures.A. Geometry of quantum computationGeometry concepts have been widely discussed in formulat-ing quantum mechanics and quantum information processingsystems, including the geometry of quantum states and theirevolution[2][3][4], the geometry of entanglement[5][6] andalso the relationship between the spacetime structure and thegeometry of quantum states[7][8][9].In [10][11] a geometric framework was proposed for thecomplexity of quantum computations. Its basic idea is tointroduce a Riemannian metric to the space of n-qubit uni-tary operators so that the quantum computation complexitybecomes a geometric concept as given by the slogan quantumcomputation as free falling. The key components of thisframework can be summarized as follows.• An algorithm of a n-qubit quantum computation systemis a unitary operation U ∈ U(2 n ), which can evolve then-qubit initial state |ψ ini i = |00...0i to the f inal state|ψ fin i = U|ψi ini . U(2 n ) is the space of the unitaryoperations of n-qubits, which is both a manifold and aLie group.• Any physical realization of the algorithm U is a curveU(t) ∈ U(2 n ),t ∈ [0,1] with U(0) = I,U(1) = U,where I is the identity operator. A smooth curve U(t)can be achieved by a Hamiltonian H(t) so that˙U(t) =−iH(t)U(t), where H(t) is in the tangent space ofU(2 n ) at U(t) and also an element of the Lie algebrau(2 n ).arXiv:1710.10784v1 [cs.LG] 30 Oct 2017
