II. Earth Learn: A Planetary Prodigy Comes to Her/His Own Knowledge
1. AI: A Survey of Deep Neural Network Learning
This is a new section posted in 2017 to survey and report this “artificial intelligence” or AI turn from a machine computation scheme to new understandings that biological brains as they fluidly form, think, perceive, and gain knowledge can provide a much better model. This revolution parallels and is informed by neuroscience findings of cerebral node/link modularity, net communities, rich hubs, multiplex dynamics of neuron/synapse topologies and emergent cognizance. A prime quality is their self-organized, critically poised, self-corrective, iterative education, and especially their achievement of pattern recognition, as we people do so well. “Deep” means several interactive network layers or phases are in effect, rather than just one level or “shallow” AI.
Can We Build a Brain?. www.pbs.org/video/nova-wonders-can-we-build-a-brain-j53aqg. This April 2018 NOVA Wonders program provides a brilliant introductory survey of these active frontiers of Artificial Intelligence and Deep Neural Network Learning. An extraordinary array of contributors such as Fei Fei Li, Christoph Koch, Rodney Brooks, DeepMind experts, to cite a few, and especially Rana El Kaliouby, reveal a grand project with immense promise for peoples and planet if it can be respectfully guided and carried forth.
Information-Theoretic Approaches in Deep Learning. www.mdpi.com/journal/entropy/special_issues/deep_learning. This page is an announcement about a special issue planned for the popular online MDPI Entropy site, which is open for manuscripts until December 2018. It is conceived and edited by Deniz Gencaga, a Antalya Bilim University, Turkey, professor of electrical engineering.
Deep Learning (DL) has revolutionized machine learning especially in the last decade. As a benefit of this unprecedented development, we are capable of working with very large Neural Networks (NNs), composed of multiple layers (Deep Neural Networks), in many applications, such as object recognition-detection, speech recognition and natural language processing. Although many Convolutive Neural Network (CNN) and Recurrent Neural Network (RNN) based algorithms have been proposed, a comprehensive theoretical understanding of DNNs remains to be a major research area. Recently, we have seen an increase in the number of approaches that are based on information-theoretic concepts, such as Mutual Information. In this Special Issue, we would like to collect papers focusing on both the theory and applications of information-theoretic approaches for Deep Learning. The application areas are diverse and some of them include object tracking/detection, speech recognition, natural language processing, neuroscience, bioinformatics, engineering, finance, astronomy, and Earth and space sciences.
Power and Limits of Artificial Intelligence. www.pas.va/content/accademia/en/publications/scriptavaria/artificial_intelligence. A site for the Proceedings of a Pontifical Academy of Sciences workshop held in late 2016 on this advance and concern. A premier array of neuroscience and computer scientists such as Stanislas Dehaene, Wolf Singer, Yann LeCun, Patricia Churchland, Demis Hassabis, and Elizabeth Spelke spoke, whose presentations both in video and text are available on this site. Search also Dehaene 2017 for a major paper in Science (358/486) as a follow up on his talk and this event.
Aggarwal, Charu. Neural Networks and Deep Learning. International: Springer, 2018. The IBM Watson Center senior research member provides a latest copious textbook for this active revolution. Ten chapters go from the AI machine advance to brain and behavior based methods, onto features such as training, regularization, linear/logistic regression, matrix factorization, along with neural Turing machines, Kohonen self-organizing maps, recurrent and convolutional nets.
Aragon-Calvo, MIguel. Classifying the Large Scale Structure of the Universe with Deep Neural Networks. arXiv:1804.00816. We cite this posting by a National Autonomous University of Mexico astronomer as an example of how such novel brain-based methods are being applied to even quantify these celestial reaches. By this work and many similar entries, might our Earthwise sapiensphere be perceived as collectively beginning to quantify the whole multiverse? Could it also allude a sense of an affine nature as a cerebral, connectome cosmos? See also, e.g., An Algorithm for the Rotation Count of Pulsars at 1802.0721.
Bahri, Yasaman, et al. Statistical Mechanics of Deep Learning. Annual Review of Condensed Matter Physics. 11/501, 2020. Google Brain and Stanford University researchers scope out ways to root neural-like networks as they come pervade and apply everywhere into an increasingly conducive physical phenomena. We add that an implication might be a nascent sense of a cerebral cosmos trying to achieve its self-witness and re-presentation via our globally capacious intellect.
The recent success of deep neural networks in machine learning raises deep questions about underlying theoretical principles. We methods of interactive physical analysis rooted in statistical mechanics which have begun to yield conceptual connections between deep learning and diverse physical and mathematical topics, including random landscapes, spin glasses, jamming, dynamical phase transitions, chaos, Riemannian geometry, random matrix theory, free probability, and nonequilibrium phases. (Abstract excerpt)
Baldi, Pierre. Deep Learning in Biomedical Data Science. Annual Review of Biomedical Data Science. Vol. 1, 2018. A UC Irvine, School of Information and Computer Sciences, Institute for Genomics and Bioinformatics, researcher introduces ways that artificial neural network advances can serve pattern finding and diagnostic needs across many realms of big biological and medical data analysis and synthesis.
Since the 1980s, deep learning and biomedical data have been coevolving and feeding each other. The breadth, complexity, and rapidly expanding size of biomedical data have stimulated the development of novel deep learning methods, and application of these methods to biomedical data have led to scientific discoveries and practical solutions. This overview provides technical and historical pointers to the field, and surveys current applications of deep learning to biomedical data organized around five subareas, roughly of increasing spatial scale: chemoinformatics, proteomics, genomics and transcriptomics, biomedical imaging, and health care. (Abstract)
Bausch, Johannes and Felix Leditsky. Quantum Codes from Neural Networks. New Journal of Physics. 22/023005, 2020. We cite this paper by Cambridge University and University of Colorado computational physicists as a gppd instance of how readily cerebral architectures can be effectively applied acrpss far-removed domains. These common transfers open another window upon a universal, iconic bipartite (node/link) and triune (whole brain, genome, etc.) nature.
We examine the usefulness of applying neural networks as a variational state ansatz (approach) for many-body quantum systems for quantum information-processing tasks. In the neural network state, the complex amplitude function of a quantum state is computed. The resulting multipartite entanglement structure can describe the unitary dynamics of physical systems of interest. Here we show that neural networks can efficiently represent quantum codes for information transmission. Our main points are: a) Neural networks yield quantum codes with high coherent information for two important quantum channels, b) For the depolarizing channel, they find the best repetition codes and, c) Neural networks cam represent a special type of quantum error-correcting codes. (Abstract excerpt)
Beer, Kerstin, et al. Efficient Learning for Deep Quantum Neural Networks. arXiv:1902.10445. Leibniz University Hannover physicists describe how readily our own analytic cerebral topologies can be adapted to and availed in this basic physical realm, especially for computational methods. Google the above institute to learn about the authors and how their Light and Matter at the Quantum Frontier project is auguring to begin a new natural creation (let there be light a second time).
Neural networks enjoy widespread success in both research and industry and, with the imminent advent of quantum technology, it is now a crucial challenge to design neural networks for fully quantum learning tasks. Here we propose the use of neurons as a building block for quantum feed-forward networks capable of universal computation. We describe the efficient training of these networks using fidelity as a cost function and provide both classical and efficient quantum implementations. Our method allows for fast optimisation with reduced memory requirements such that the number of qudits required scales with only the width. (Abstract edits)
Beny, Cedric. Deep Learning and the Renormalization Group. arXiv:1301.3124. We cite this instance by a physicist at Leibniz University, Hanover, in 2013, presently at Hanyang University, Seoul, to record how quantum information, the title theory, complex neural networks, computational methods, and more are converging in a fertile process of cross-correlation.
Renormalization group (RG) methods, which model the way in which the effective behavior of a system depends on the scale at which it is observed, are key to modern condensed-matter theory and particle physics. We compare the ideas behind the RG on the one hand and deep machine learning on the other, where depth and scale play a similar role. In order to illustrate this connection, we review a recent numerical method based on the RG - the multiscale entanglement renormalization ansatz (MERA) - and show how it can be converted into a learning algorithm based on a generative hierarchical Bayesian network model. Under the assumption - common in physics - that the distribution to be learned is fully characterized by local correlations, this algorithm involves only explicit evaluation of probabilities, hence doing away with sampling. (Abstract)
Biamonte, Jacob, et al. Quantum Machine Learning. arXiv:1611.09347. A six member team with postings in Malta, Canada, Spain, Sweden, Germany, and the USA, including Seth Lloyd, advance a novel synthesis of recurrent neural net machine processes with quantum phenomena seen to possess algorithmic, complex dynamic system, information processing affinities. (Quantum Complex Systems)
Carleo, Giuseppe and Mathias Troyer. Solving the Quantum Many-Body Problem with Artificial Neural Networks. Science. 355/602, 2017. As the Abstracts notes, ETH Zurich physicists find this generic iterative approach, as it gains utility in many areas, to be an apt method for dealing with and solving such seemingly intractable phenomena. The work merited a report in the same issue as Machine Learning for Quantum Physics (355/580).
The challenge posed by the many-body problem in quantum physics originates from the difficulty of describing the nontrivial correlations encoded in the exponential complexity of the many-body wave function. Here we demonstrate that systematic machine learning of the wave function can reduce this complexity to a tractable computational form for some notable cases of physical interest. We introduce a variational representation of quantum states based on artificial neural networks with a variable number of hidden neurons. A reinforcement-learning scheme we demonstrate is capable of both finding the ground state and describing the unitary time evolution of complex interacting quantum systems. (Abstract)